You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/03/27 07:02:00 UTC
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #311
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/311/display/redirect?page=changes>
Changes:
[boyuanz] Remove TimeSpec from proto
------------------------------------------
[...truncated 279.88 KB...]
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 27, 2020 6:58:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 6:58:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 6:58:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 6:58:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 6:58:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 27, 2020 6:58:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-pQglYe_e7E5HyL7pfDvscQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-XngBd91mySGeljhPWCdGmQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-AXwxMBPTDv2l-9TTcZo9IQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-ZpynhFwfSBXBPceC_lbPAA.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-WQ6d0K0oS82dlFuC-Yt0sg.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-uE1NdlqUgE5cFSj8LPCHcg.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-6La7fp27S0qti_wqEmE2AQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-ELYAhHAdLxppzRUPpZ2WAQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-rQW720AaqyaIPwv2CIdEcQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-DVQoXh1Eo8GzPPJksAsbxw.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-zVFe_K8430UwOWVcBdk-LQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT--YFWWCOnhZvA5IuiIAQIwQ.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-sy6vgGhbfIFYk1QVBnQAFg.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-G3K-uwTN50NiYexmar58ww.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-saQWbMCzp6iAO0_Of_PfwA.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-aF4QffWTrX8z4z0ByJrJ8w.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-a7OAo61Wixi6iZftbkWxMw.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-O6vQocZGdRzI71FzBSrVeg.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-tNjGsmdXZKlA_wgBPWe2pw.jar
Mar 27, 2020 6:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-dbA2XCvmncexF8GlBVAESA.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-znJTeZQzFvlRlKZWd6iHMQ.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-rSU09Lsa9ckPEmfXnXmdww.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-PD_Faq_9t_6dK89nuwbeZQ.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-mqkkxONgDIS_BXn8NlI52A.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-Oy8lcEm8n3ep_ARN513CIA.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-zYc_bSiE8RP2cujrTXfXVg.jar
Mar 27, 2020 6:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-OlW5HCBDM9yCgtA8FTX47A.jar
Mar 27, 2020 6:58:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 27, 2020 6:58:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash FTt_0OoqtZ5JsHV3Ye7Zaw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-FTt_0OoqtZ5JsHV3Ye7Zaw.pb
Mar 27, 2020 6:58:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 27, 2020 6:58:36 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 27, 2020 6:58:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-26_23_58_36-4880206109906800179?project=apache-beam-testing
Mar 27, 2020 6:58:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-26_23_58_36-4880206109906800179
Mar 27, 2020 6:58:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-26_23_58_36-4880206109906800179
Mar 27, 2020 6:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T06:58:36.230Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 27, 2020 6:58:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:40.134Z: Checking permissions granted to controller Service Account.
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:46.317Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.017Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.051Z: Expanding GroupByKey operations into optimizable parts.
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.086Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.178Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.219Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.257Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.298Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.764Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 6:58:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:58:47.935Z: Starting 5 workers in us-central1-f...
Mar 27, 2020 6:59:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:15.524Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Mar 27, 2020 6:59:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T06:59:15.548Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 27, 2020 6:59:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:15.563Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Mar 27, 2020 6:59:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:20.955Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Mar 27, 2020 6:59:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:21.126Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Mar 27, 2020 6:59:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:42.029Z: Workers have started successfully.
Mar 27, 2020 6:59:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T06:59:42.071Z: Workers have started successfully.
Mar 27, 2020 7:00:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:00:04.277Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running step(s).
Mar 27, 2020 7:00:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:00:11.100Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 7:00:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:00:11.334Z: Cleaning up.
Mar 27, 2020 7:00:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:00:11.418Z: Stopping worker pool...
Mar 27, 2020 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:01:48.710Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 27, 2020 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T07:01:48.782Z: Worker pool stopped.
Mar 27, 2020 7:01:56 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-26_23_58_36-4880206109906800179 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a05a5681-7142-44a6-bd9d-fe45510a7618 and timestamp: 2020-03-27T07:01:56.086000000Z:
Metric: Value:
fields_read 4375276.0
read_time 10.89
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 33.411 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 39s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/nxc3sp3vtllxy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/561/display/redirect>
Changes:
------------------------------------------
[...truncated 294.86 KB...]
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 29, 2020 6:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 29, 2020 6:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 29, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 29, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 29, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 29, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-6uTiODw5KzPpaULAZhiqQNpBJV4fUfhSX13DW_WaLJM.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-Hox9BkamGlFXIB2Cgo8ud1q5-GJutpkkqV0tIFBCja8.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-5pxFJgo9dZ3kzk3fQGx9GJE9HZp7MtZCtCyhaqzdeAY.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-7KRJjV49tU94Y8q6F-4WZramxmX1FOX3H2du8PgN3-Q.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-wA-SsLjdGegBiSkLHzL9iQhTigTsYkXmscmxw2ruBM8.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-7e3_ACWoqx_d13BYUlY2bVegLkuz8T38uFLJvT5mw8Q.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sLfuEXjDDxLyPMiEqB4QxHqEVrSFVjrgtQqJhibPBa8.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-k9DnEeYX44eGL0QtdIVr_fYCGP5CuAzffyztUg88z1w.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-YEEwDIpU_Hcvg5t7EDjLVscjQz_m4DXqU4JLBFq08nc.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-IYlowzisbua0y6qJp1BisCVBTUcoYzidrTGg5jR5WBg.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-4RfRECKh3lDjfnweoX4449kjQkZ1hoN-ujt7VvAb-oc.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-bqIY0wvaMkTyKxDQciVGf3WHqqDuGFm_i6ICetlRTP0.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-H-gjfXyCK3TdC7aM5OZtFVNtBGoAAL1ccHPKOUGE148.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-ktrae5U-1u07c0MhuILKtXB1BnByU2W9D1fJLh-gcsI.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-9PLOYAR0-EpZaqyKSa9FpnG0kcWdFWqSNntTRI5vFxQ.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-cU2_lHHLWYjLLv8KM3OBUt5aaJ4pbm2SAFKgILl1i5U.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-00ySORHpn1Gc5PrQTIAnPrtBvPSQzpDPDoOeFHbdH6s.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-Nnbmr8bnLxHJnRb6kATaQlwoVlxeN9_vwUv-2aY7wSE.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test4968701269091602709.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-58I1u3-NMV-s7_s_9DHJmpxEMUndNi1O3Yf69Xu7UAk.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-ZlOVgrtGHMHERLLutc5aHMut01kO4rffeyxWwDaZoEo.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-hMdPy6nTxrtKmu0nTRK8mVopkA74n70mGI5NgOz1WWk.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-57ndYDdAtvkx9DF5L3nKknKTlyYh1jycQpufGr6KplQ.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-5gHA9j2CQ0FUedicvL1tnumach7lCnQ-tnoH_Oo6W0Q.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-NTXBupKz6osYGY-R_kVyE3I2uUjeFZe_j4xU1GUegAA.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-MBJhpTBBJ1--cPuKyB1Mku50tk9lgPkuGGPLw-wb2Ug.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT--feKqa6_KY48HVT7911z3VQVcx1tUtZOZJgjev3-lho.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-wOfwYuiy85yyLJPZhBvAIQNbLZkjN-3UZbD0vLyn9hw.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-eJdeVAwKvpjUG4SjCshUDVCSamhed46nzmHHDeYTIK4.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-IsgN6Tn0AlaeVkREjxUmdrAki4JFslrOPBiZ-8oLgQA.jar
May 29, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-Mf0MOLjXeeiGSVJ0ekRYYZYPhCjbWeFhioTolP_X058.jar
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 1 seconds
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 29, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 29, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91698 bytes, hash 7bb936d1c18202c3ee8039c0832e11735beaca16bd9865765efb1da6f4ca8599> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-e7k20cGCAsPugDnAgy4Rc1vqyha9mGV2XvsdpvTKhZk.pb
May 29, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 29, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-28_23_45_40-3013672068961674668?project=apache-beam-testing
May 29, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-28_23_45_40-3013672068961674668
May 29, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-28_23_45_40-3013672068961674668
May 29, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-29T06:45:40.588Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:47.486Z: Worker configuration: n1-standard-1 in us-central1-a.
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.252Z: Expanding CoGroupByKey operations into optimizable parts.
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.288Z: Expanding GroupByKey operations into optimizable parts.
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.316Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.394Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.429Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.462Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.483Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.897Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 29, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:45:48.966Z: Starting 5 workers in us-central1-a...
May 29, 2020 6:46:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-29T06:46:07.477Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 29, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:46:16.573Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 29, 2020 6:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:46:33.463Z: Workers have started successfully.
May 29, 2020 6:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:46:33.500Z: Workers have started successfully.
May 29, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2020-05-29T06:46:51.515Z: java.io.IOException: Failed to start reading from source: name: "projects/apache-beam-testing/locations/us/streams/CAUSDHYyM0h2UGV3RDZzcBoCaXIaAmpkGgJuYRoCb3MaAmluGgJqcRoCanIaAmpjGgJvahoCaXcoAQ"
at org.apache.beam.runners.dataflow.worker.WorkerCustomSources$BoundedReaderIterator.start(WorkerCustomSources.java:604)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation$SynchronizedReaderIterator.start(ReadOperation.java:361)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:194)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:159)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:77)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:417)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:386)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:311)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: there was an error operating on 'projects/apache-beam-testing/locations/us/streams/CAUSDHYyM0h2UGV3RDZzcBoCaXIaAmpkGgJuYRoCb3MaAmluGgJqcRoCanIaAmpjGgJvahoCaXcoAQ': internal error
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:86)
at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
Suppressed: java.lang.RuntimeException: Asynchronous task failed
at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageStreamSource$BigQueryStorageStreamReader.readNextRecord(BigQueryStorageStreamSource.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageStreamSource$BigQueryStorageStreamReader.start(BigQueryStorageStreamSource.java:205)
at org.apache.beam.runners.dataflow.worker.WorkerCustomSources$BoundedReaderIterator.start(WorkerCustomSources.java:601)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation$SynchronizedReaderIterator.start(ReadOperation.java:361)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:194)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:159)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:77)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:417)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:386)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:311)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: io.grpc.StatusRuntimeException: INTERNAL: there was an error operating on 'projects/apache-beam-testing/locations/us/streams/CAUSDHYyM0h2UGV3RDZzcBoCaXIaAmpkGgJuYRoCb3MaAmluGgJqcRoCanIaAmpjGgJvahoCaXcoAQ': internal error
at io.grpc.Status.asRuntimeException(Status.java:533)
... 16 more
May 29, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:47:09.633Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 29, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:47:09.844Z: Cleaning up.
May 29, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:47:09.924Z: Stopping worker pool...
May 29, 2020 6:48:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:48:04.628Z: Autoscaling: Resized worker pool from 5 to 0.
May 29, 2020 6:48:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T06:48:04.677Z: Worker pool stopped.
May 29, 2020 6:48:11 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-28_23_45_40-3013672068961674668 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ae404a82-2270-4b87-a847-b1742c1170c4 and timestamp: 2020-05-29T06:48:11.353000000Z:
Metric: Value:
read_time 16.333
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 29, 2020 6:48:11 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 2 mins 44.971 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 54s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/dxacjeadjgdwc
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/560/display/redirect?page=changes>
Changes:
[chamikaramj] Enables proto holders for testing runner API. Without this x-lang can be
[github] [BEAM-9679] Flatten Kata for Go (#11806)
[github] [BEAM-9220] Go Dataflow jobs to use runner v2 (#11207)
[robertwb] Update multi-language roadmap status.
[daniel.o.programmer] [BEAM-9935] Respect allowed split points and fraction in Go.
[github] [BEAM-8280] Blog post: Python typing changes (#11070)
[github] Update website/www/site/content/en/roadmap/connectors-multi-sdk.md
------------------------------------------
[...truncated 291.36 KB...]
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 29, 2020 2:08:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 29, 2020 2:08:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-OUKLw0E6jqbOMEX2V64tFJ2YEjvcHJs2kYzGpqCyb1U.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-7WPms-EImphB0-1EPZZ4ygj8-qzfRRPuNhKAWZAkdrc.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-DbSMVfPZrEcjDJpc87u9wCPYNlmtJU0qWghAi5b5lL4.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-5Tr2V47Spf86sp7Gyo-Y7Gp606G4p2O0eMiD5PdI2eg.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-9zptZJTt-zhLkeQEtwKumhhTFKANkg2kbzFZ6sdViu0.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-L3EruqMfkiiQVWlpAUNt_PmJC3CMzDG6kM3T1cc0-VU.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test7124792243384836422.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-OuI3KkIBbuHSQhJt_FbvL8m78_Lvhud7VNeTGpUft9E.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-oi4Sd0tj-IS-FAsQo60NGIz9Aq2-vZS7AB-AFnDGZzg.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-IduTEirVQaf2Orh4BFuS0SLbLIbhCDHQMxr6RKsmdTU.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-YMDgbs79tPjiw6rQkBUzy5KSXNVhPMcPm7uR4nhDn5I.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-dXLqFc9DCLCU4cZgJwPeIvg9IeSrBFoMO7o_K592l3s.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-Q-NgFkDbADM0vEQKfCseqW2LGBZLZzkI4kGjusOUDes.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-_F4YZF515_cC504rbm2g6bh9rsDLRtrx0ZYqddwClek.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-YhBGUQCUylRTN5GaLhfoVDqBKh4ljaPE0NYQbOW_iYI.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-L3B5eaJuUr_vHdJ3oWQk5HBKDUk48Env6q0nDKqYsmg.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-cof4SN3inr-DnUapP4LonjY8s-juvmBPRj-JuYefjf4.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-YBcPdTshy_H_mhq48t1TtktQPGBZIErwE2fshWUdUUE.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-DRQFo2Vy5PbBDsYhPKoRKpk-uqCT_UFfraB9kBbSAAc.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-jF49QNZiP3O4tkqX75hgchyqR1jeGyhzAaqS2ing1qU.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-upn__gYCQYjsBlqW_j85hLE5cOiBiPm1QPjLNfwAzXs.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-NMoziZivKP92lW6KzAYI222hbgDCyTGXtWoyLj8eucs.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-g2dD2fi-gdC7KinhIH7WJlemeGAfppA8g-2mstYeqcE.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT--5NCNg5J2VXrsG2k7zSvDiEKH2xBpFU-kqJ700ky0O0.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-IG6PCpOuwGE3-mwja-KbqwBPTglvtWb35Ys6_baYCsg.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-gowzEGj5f5k4dXvgcbc2cEnTAUY7N66kuVVJG5sx4bg.jar
May 29, 2020 2:08:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-1Ie96qkdrck82yO2CFSpXcRsxWT7PKYRiKUtXiEs77E.jar
May 29, 2020 2:08:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-n8gd4pCaQa3rF9iFO5dN8xhk-3a4PflIvtptnWXV7Vo.jar
May 29, 2020 2:08:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-MXjHPJzIyPCL3oppYTKiEzeA8WR_AhLTj3DsHsQuUDM.jar
May 29, 2020 2:08:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-ujZOfjYea1MGny79UpByV7TRed3Gd0sFA9mJNi2xdik.jar
May 29, 2020 2:08:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-AP39A4oZJWTe0KZBC3j5v-mqpXPxW6mEQ_9ZFa_z0G4.jar
May 29, 2020 2:08:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 2 seconds
May 29, 2020 2:08:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91698 bytes, hash dc250b5f6f1b27970d16e88c69f2a77d3a4459e6a9c64e34ba517b42df0166e0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3CULX28bJ5cNFuiMafKnfTpEWeapxk40ulF7Qt8BZuA.pb
May 29, 2020 2:08:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 29, 2020 2:09:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-28_19_08_58-5397891860805293169?project=apache-beam-testing
May 29, 2020 2:09:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-28_19_08_58-5397891860805293169
May 29, 2020 2:09:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-28_19_08_58-5397891860805293169
May 29, 2020 2:09:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-29T02:08:58.975Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 29, 2020 2:09:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:07.242Z: Worker configuration: n1-standard-1 in us-central1-a.
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.101Z: Expanding CoGroupByKey operations into optimizable parts.
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.147Z: Expanding GroupByKey operations into optimizable parts.
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.175Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.282Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.327Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.359Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.392Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:08.945Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 29, 2020 2:09:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:09.053Z: Starting 5 workers in us-central1-a...
May 29, 2020 2:09:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:32.928Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 29, 2020 2:09:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:32.966Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 29, 2020 2:09:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-29T02:09:37.027Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 29, 2020 2:09:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:38.377Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 29, 2020 2:09:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:49.634Z: Workers have started successfully.
May 29, 2020 2:09:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:09:49.667Z: Workers have started successfully.
May 29, 2020 2:10:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:10:28.137Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 29, 2020 2:10:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:10:28.359Z: Cleaning up.
May 29, 2020 2:10:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:10:28.444Z: Stopping worker pool...
May 29, 2020 2:11:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:11:13.733Z: Autoscaling: Resized worker pool from 5 to 0.
May 29, 2020 2:11:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-29T02:11:13.795Z: Worker pool stopped.
May 29, 2020 2:11:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-28_19_08_58-5397891860805293169 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3ad24d54-d0c6-46ce-987a-200651960371 and timestamp: 2020-05-29T02:11:19.168000000Z:
Metric: Value:
read_time 20.213
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 29, 2020 2:11:19 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 2 mins 35.43 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 50s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/sawgi7km3refs
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/559/display/redirect?page=changes>
Changes:
[kmjung] Reject fractional values outside of (0.0, 1.0)
[aramaswamis] [BEAM-10067] Minify website assets with --minify flag
[altay] Add a bare bones powered by page.
[kcweaver] [BEAM-10106] Script the deployment of artifacts to pypi
[kcweaver] Add pypi script to release guide.
[kcweaver] Improve log messages in release scripts.
[kcweaver] Install twine in virtual env.
[github] Avoid re-encoding value state cells each commit (#11823)
------------------------------------------
[...truncated 287.73 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 6:55:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 6:55:55 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 28, 2020 6:55:55 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 28, 2020 6:55:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 28, 2020 6:55:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-nWlQ6iS3Rq1idvLClbKaRAzS1VYX4Jn37MP8pEIUKnk.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-a38Oby2m9CSCn9xAP_lbGCdT9vNL2fJzdvyU-J-i9UA.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-gAFqu1AX-3ASqieGKE-dnjCga3HykEzcsKSR5mHyUWM.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-QHTcsfQHGdtDBF6JMI5Eg0CzvXqidSxtV7lsvWNzxRI.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-iLu2GSnXYlvPtlOsyuNbKcOLZGEC-LVF2ul6eM9TS-Y.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-8TUf41OvcugDrrMzlKaJZp9u3JIvxW8d6Tqk2OXhzfk.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-wHWogaMk2Fw0ApMEACJBCZAWcJVLHzoyiUs6WU9rs9U.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-8q3OHOLo7M2r489StQANxso1a6aicGi09DQCLKt9axQ.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-r4HMiPYVgj6_zFIDJKAhfXfj8C6j1dXiq8HAqjwBhfs.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-sGC6xeoPiF6c3IjP6hQqwl8DFIopG6uOIUK0WvPOS1Y.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Clz0v9Pu4YSdSYK8PBpIZAkaBAEfbHT0EaeMegwTxfM.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-BVqynAxUrkA6Zo_NIydpzNiyo4E8cmsFsg4uqYmbD3I.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-7mOICs_6ImtvBKXVDPT_alyahN-cdEA-_Muz5QA1uwk.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test4549113455894754623.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-2Hk1F6Bxuowd9h1826hiuYdMnPGsmX9-rFhhCpsh-TA.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-hbfCjlFO2seQnFhOhvWPwvQCjvncm9D0Cq2myWLLM_g.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-3dmzaRVCfMSd311EkRL7nLy2NnA2sFmzhdrQD9wm--Q.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-h_DvKz2jok3_errXIbwNZ-8uYJZQXJkhUNBwgKd6PKk.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-JQA7qehoYnYsWAuEHqpTEpzdJaUAqJH0dcJyMin8JOI.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-aKuDAFg06WfNFIsgiF8Es5B3-ETOnNzep8IgdGCl86Q.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-_Z49lOr34eDaphT0X9S7zvt1fQUNvt2CBbt7xTUVC2A.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-C93oSVQ_sUTszW1Fp44Admk24RAgmOY3CjaVVzeRJPk.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-x-DqnVhS7zfIHO3BsbkyBs1_PopCRkPzpcI5pfy4El8.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-yZyF7aoUmEtiTrRrNzdYJa05D7y8Fs4-Wt8Snt-jMO8.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-DOq8HlNqfQsqOYeRlTdkg4rdt0Et9PziK3oEboPzwYE.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-xUqmOykR4776I0WPFbQ7k0RUpd3w5mT0Xi27_HbLYHQ.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-Sg8hnJh-v1_22eu63AcEPAVavRmjzrwjE3hF7CowcNw.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-34uLnm7VAAo-I_uE54A9XYgQEsXPgIexOkNT_PiDETY.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-Pvb3YyeLEdROONXVwzrD567nEibR6SAhD6NS-Paoek8.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-CLU1cL0os3CRTHwSvGbfeRrNKtDn5PS2zBLFVFvzMVU.jar
May 28, 2020 6:55:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-Fu6rA1Kvn8kwzRsjcGYj2tYSEIjBebTNL3r1yZy3jnM.jar
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 1 seconds
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 28, 2020 6:55:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 28, 2020 6:56:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91698 bytes, hash 52f3558e34de7979a0b54b2e06c1a08d74285da565423861c376e9237594a018> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UvNVjjTeeXmgtUsuBsGgjXQoXaVlQjhhw3bpI3WUoBg.pb
May 28, 2020 6:56:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 28, 2020 6:56:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-28_11_56_00-5025367664640387008?project=apache-beam-testing
May 28, 2020 6:56:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-28_11_56_00-5025367664640387008
May 28, 2020 6:56:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-28_11_56_00-5025367664640387008
May 28, 2020 6:56:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T18:56:00.652Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:09.022Z: Worker configuration: n1-standard-1 in us-central1-a.
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:09.845Z: Expanding CoGroupByKey operations into optimizable parts.
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:09.895Z: Expanding GroupByKey operations into optimizable parts.
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:09.929Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.014Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.054Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.092Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 28, 2020 6:56:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.129Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 28, 2020 6:56:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.700Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 6:56:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:10.792Z: Starting 5 workers in us-central1-a...
May 28, 2020 6:56:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T18:56:35.775Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 28, 2020 6:56:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:36.412Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 28, 2020 6:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:53.844Z: Workers have started successfully.
May 28, 2020 6:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:56:53.877Z: Workers have started successfully.
May 28, 2020 6:57:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:57:26.422Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 6:57:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:57:26.673Z: Cleaning up.
May 28, 2020 6:57:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:57:26.765Z: Stopping worker pool...
May 28, 2020 6:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:58:17.631Z: Autoscaling: Resized worker pool from 5 to 0.
May 28, 2020 6:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T18:58:17.669Z: Worker pool stopped.
May 28, 2020 6:58:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-28_11_56_00-5025367664640387008 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 58e84ed9-9b2a-451b-8092-5d7072151582 and timestamp: 2020-05-28T18:58:23.278000000Z:
Metric: Value:
read_time 13.917
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 6:58:23 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 2 mins 36.939 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 28s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/pidjv7o44cs7w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/558/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-10098] Enabling javadoc export for RabbitMqIO and KuduIO
------------------------------------------
[...truncated 289.21 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 28, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 28, 2020 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 28, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-TPGSYJPyXLcoGOXIyE4x64_HzJbrdlVO2xXsPFho_uc.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-1REnwhzSzH-UsI_CMcBvzQRI5mOd1QX2aIQymEAoIUc.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-kYxbmzdqtlXGwPF9PHoZ1SLjik0b0mRwM0G-zotS3AE.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-pgNhSyDgG0cre94SRuKS2AUv3DpIEP-sEJiEM8I9j9M.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test823668107536180387.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-hHhPYQrQMxX9Qvk2ttgj2A6QF-b1aiOJcHF7ps00_H4.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-1jnDaweYLA3onJg-b3X2aUOWRaWRPhoHiYJjsgwBZDw.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-4i-E80GDonkK2VWl4ZkJQc73j-HygNbE9ExdOob6iZs.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-9hLBrnwysmMA2jQxyh27H5uA1k8ndPcKl7Ww9v144cs.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-QFReNnjWSSA3iex3na_vxq1zZMR7GyxB01PsTnCYZws.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-RWfoKACqSRUf2Wxo9RT2YgiflB7EIXBVx2EJM0RBpxA.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-mIvB8JkpOBFi65VqFHQkWGUUoSRQ-a5HQGMbR_8yOYg.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-1CDEKZmbJ1tsJSIPsYxYp2Q528tBTVCfC3l9v471IMY.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-m7ARwbaIY-5x5N1Q3EKXYhWlTpLpvulaKly6ALuEt4s.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-NOXsHjydG1Dkw8rtuk-E5PUHcUCQeT5fjUXlo6HATn8.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-eXU9wOuN8qmwGThb9Lai1DuuVOWMO3UaF2bXe69H1To.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-UYCQZ9nQh9W5eXxnmZWYESf58Je_t5_1Nmw0TSwB_FU.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-C9kPWolMPF2U6GMCNMlrftdHiM-Ehi4PpLbHiZlBCPo.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-oUAv_6UrkZPA95-9yP6SFUZNBXN9sXemOO1plLDj518.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-0Ttez3-X8M8mtMfupnu8VdHcaRIIFTG8VkwlDJhX6jc.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-LObAZPua3VSVLP0NZOaC2qrYUyPoRlgtE_ve3PE94Q8.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-aCkc3JUPZ_H_TJKCo8l85gPMLhQIJvkmki5dOrukvZQ.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-kjsuzF29aFYq9P3UhOv8d5wPlKbKzM95liWVrnfYa_M.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-7BE6QqYAMh154_sSn66UGlZcC4mUnQxzQQLdsWutK1s.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-o8R_azU71QAdTpRspQvuK2F2B1RRKQXANkX7CzS_FpU.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-kGfx0SD1zsbmT5STF35LABpcvef6MhPANwt36g-_SZ0.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-GvF7vxodAehdST5qyBXbtIOfbKqZqdX86yu1Mwj52jo.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-nRSBKYeSlReKW1Fr48TgUklwcFMNj_sB88qbAreXt7Q.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-viyTu60yhsZFa5nwQwjUhmnz_7IdSkQeuHBB4TEC5qI.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-qM_-HBTmXDxrY0GkB5uKCjmoT6lIf3Mw9RTdDb-Bkbs.jar
May 28, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-_5FxtUjYjwYusAbS65MRUQmJhOGnkZi1ooitHsLxyLw.jar
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 1 seconds
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 28, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91697 bytes, hash 5e0f335ee26cb275b8f3d49f9664c3fbe06255b74258a09d84a9c77b82b89b27> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Xg8zXuJssnW489SflmTD--BiVbdCWKCdhKnHe4K4myc.pb
May 28, 2020 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 28, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-28_05_45_40-10048155393749943304?project=apache-beam-testing
May 28, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-28_05_45_40-10048155393749943304
May 28, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-28_05_45_40-10048155393749943304
May 28, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T12:45:40.347Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 28, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:49.195Z: Worker configuration: n1-standard-1 in us-central1-a.
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:49.842Z: Expanding CoGroupByKey operations into optimizable parts.
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:49.886Z: Expanding GroupByKey operations into optimizable parts.
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:49.920Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.007Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.039Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.078Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.103Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.722Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:45:50.785Z: Starting 5 workers in us-central1-a...
May 28, 2020 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T12:46:12.831Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 28, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:46:25.894Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 28, 2020 12:46:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:46:44.958Z: Workers have started successfully.
May 28, 2020 12:46:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:46:45.008Z: Workers have started successfully.
May 28, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:47:19.011Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:47:19.203Z: Cleaning up.
May 28, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:47:19.288Z: Stopping worker pool...
May 28, 2020 12:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:48:10.888Z: Autoscaling: Resized worker pool from 5 to 0.
May 28, 2020 12:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T12:48:10.931Z: Worker pool stopped.
May 28, 2020 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-28_05_45_40-10048155393749943304 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): aba801e6-7af7-4a32-be61-0192d6185757 and timestamp: 2020-05-28T12:48:20.090000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.927
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 12:48:20 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 2 mins 53.854 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 2s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/fxsbhgt4vjt2y
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/557/display/redirect?page=changes>
Changes:
[github] [BEAM-9363] Support TUMBLE aggregation (#11807)
------------------------------------------
[...truncated 289.43 KB...]
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 28, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 28, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 28, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 28, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-32kN6zTnp0dYGDG-M2jnuJpMlZ00l8Wq5oblnBdPki8.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-zzV8VXIPq-fl4CiAI9441nho2_aRzG-x_LPr5tTAAbY.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-CP0sasBXnYcfM_QqscdYW-rjelQVsPy4w3YDW7em0mc.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-fGVs7Mm3VSUqdWOPuZU6EOUBUUR2jum9dwzRngYdi3E.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-ZHA37frGqawlPjDos8m3YV2gZzAXhEiW1pSDluiBHVU.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-e9LPTU6HrozCzISyQaKOK50_UcU6IsVEVR7Y8koiq7s.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-_0tcBw7mSkP5gMdhI9IhLp8FtmngZaOclS1QHOeYPZw.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-WecgFl6wjYDThVsFB8BqzF3u0rQYsYyT19ldlJjnsOw.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-660rBy-untMiBojqCCVQD6qmKO9mjUs4w4snMZcFbfk.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test4239981842883870885.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-W_FDQZeehqoiuwAMjmrCXcTnzWv2IsPulIFGFKgJ39A.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-O7_FIGZkEF7ip4UEYXtHhKezLCzbUyiHlkttWeGt-CY.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-S6Za7OMhYNTZ42-ECKImm_MC_-HN0sXuIAJESit4nHU.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-tPOXpb2h-NXP-tzrXGquYaC_lG1vtpOv-twy-E5TqGo.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-_UwnqgDvdagTf7ZESNLo0BlK_Jcf_IFnj0VUtzE63OY.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-3BmJfjERuWP-B7R93PClfeMZBOO3VDWZVJs6I10DmdQ.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-ulo_z_cbhwYzAmZ-yaNnFBBkNS4W-wKGdo3qxKw6mHY.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-2L4kNUbyTklTL0gu8KMXYz5XCirkmaeY5xmNeOdUyVI.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-tUndg0u_yN2rmiDDZrLN6C9F-ow6fdFu0cKoL-IOs4Q.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-ENtcW5LpRUCo-VI-dgVRHEiiHQN9YA5XISYL6u-Ht-Y.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-q7sJbiYYRA9lEQV0n9iJfJVB40HRgMIQzZL0oSveu30.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-OKV1OiKGPAv5oOKCORZsYYhklosbu2kBd_DMd4KIaoI.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-d0ferD0EAmqp7DQwtFRDSyXDV4s3Kw61xmdgzpRSAF4.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-jRf8i4lIf-2epnTWXUzZOFZeju3geGw6x3_IULDeX3o.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-1VxuU2z1srIJBm6TiWKwNkZCtFdZcZ6ZzmMEaDA4MaM.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-nhXNe5XSPlJUwrnZ8R8kBQdHaf7Z15y0mHuJPmYBVWQ.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-wP7xVQppSf0bu2wTA18283fPueAHfi4g_DD4ytkn4bs.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-FxA8sA-HSWP6OoY6bpQDFcRNE3h9yNw2Y3K1EVWz0y4.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-xf91CcVGEMg_vhIT5eAB1Q-xgHPSqL3Iytw4M3uBGts.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-LntL3M0AMH93sVRJmTXMKJ7R1aXyVxehB43Bqh1ph2c.jar
May 28, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-N8Dla3KAf0ViOmVVP7yqnI2ZEhheemRWbcLSHEXTh-4.jar
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 1 seconds
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 28, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 28, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91702 bytes, hash 520079ba20f39909ff5523f38ec596fa5181b23e4fdcc82320a843f1a38699c9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UgB5uiDzmQn_VSPzjsWW-lGBsj5P3MgjIKhD8aOGmck.pb
May 28, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 28, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-27_23_45_45-8313561011987703739?project=apache-beam-testing
May 28, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-27_23_45_45-8313561011987703739
May 28, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-27_23_45_45-8313561011987703739
May 28, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T06:45:45.535Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:53.288Z: Worker configuration: n1-standard-1 in us-central1-a.
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:53.885Z: Expanding CoGroupByKey operations into optimizable parts.
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:53.934Z: Expanding GroupByKey operations into optimizable parts.
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.037Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.127Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.161Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.196Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.235Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.594Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:45:54.686Z: Starting 5 workers in us-central1-a...
May 28, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T06:46:03.192Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 28, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:46:18.429Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 28, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:46:18.472Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 28, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:46:23.949Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 28, 2020 6:46:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:46:38.095Z: Workers have started successfully.
May 28, 2020 6:46:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:46:38.140Z: Workers have started successfully.
May 28, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:47:11.969Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:47:12.153Z: Cleaning up.
May 28, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:47:12.234Z: Stopping worker pool...
May 28, 2020 6:48:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:48:04.385Z: Autoscaling: Resized worker pool from 5 to 0.
May 28, 2020 6:48:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T06:48:04.441Z: Worker pool stopped.
May 28, 2020 6:48:11 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-27_23_45_45-8313561011987703739 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3fdc2185-8bd7-4dd8-bcaf-826536311446 and timestamp: 2020-05-28T06:48:11.850000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.693
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 6:48:12 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 2 mins 40.272 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 52s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/sv2ecracpyywy
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/556/display/redirect?page=changes>
Changes:
[kcweaver] Update Beam website to release 2.21.0.
[kcweaver] Add blog post announcing the Beam 2.21.0 release.
[rohde.samuel] Add to/from_runner_api_parameters to WriteToBigQuery
[heejong] [BEAM-10064] Fix google3 import error for BEAM-9383
[darshanjani100] [BEAM-10074] | implement hashing functions
[kamil.wasilewski] [BEAM-9810] Tox suites for Python 3.8
[kamil.wasilewski] [BEAM-9810] Fix 'dictionary keys changed during iteration' error
[kamil.wasilewski] [BEAM-9810] Disable all failing tests until Dataflow runner supports
[github] Add release date.
[github] Add release date.
[github] Add release date.
[github] Remove HTML comment (causes formatting issues)
[kcweaver] Move 2.20 download addresses to archive.
[github] Merge pull request #11796 from [BEAM-10003] Use local code for building
[github] [BEAM-9971] Do not use context classloader. (#11784)
------------------------------------------
[...truncated 289.71 KB...]
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 28, 2020 3:49:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 3:49:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 3:49:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 28, 2020 3:49:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 28, 2020 3:49:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-Amnmb1qwq622ZN67CYawiA5jzUgdP5XDHQNWcG5PPs0.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-C_K1UFNjyMIAJQfSKkYR2mF4XVS_xNT_AG65WzsQn7E.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT--P0JvlcRdT1SzVbGoipKHdsE9CaTiANW0PS1Vk3Xg30.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test8628396184210459722.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-Mydpd1fFoWA4HrLntIVJOALwHRy0mfyBMIdjJPMMCVw.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-wjQwTGryQ8IaCSelYX84A4-aga4zCMx5AA-vdh0sIig.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-Qn60FqxYbOq6R97UdH3rzkyQ99mW3xvU9Z8y2teZkZM.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-C0fD_0Qu0itzR6wmXghJhRTVJjr-FC-_UzFUFM6FnK4.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-fm2He-gApePxMQh8h8SrHolg7nePXkgM9tM9rA-UHEo.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-ypwr9n-ybKDhqBQs-V95oUuVQsWnbvLmulmFV2fP9Mw.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-YlTOFKqJVGc885Wr1VvXmkRdWJCgRxaARAYJHZJ7lLA.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-rSefhSQNcLa4IKI7XyCt7_s3jjJny-GZOeuQ-vm6AA4.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-GgJts0tU6byZwNLC9gY2ChqezmtlgAQGJgXiAu1Y6tA.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-k8XVuFoWjcxurT4VgBK0mbEHfb1Cbra1NRG9yKsE69E.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-OiE73eAV1JlcM_wi3v5ipH6meF6PnkzUUSsEl-8-9Ug.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-qFgjwmGK7m-BBVnaPX19yMzvWu-fAk81IP08lTlkgCM.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-rKi0SQ7kyObjQPFJYZ71Qhw0BzP_OXH6ByWR8V9lra8.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-84Y25sJfzUOuGHhyYwioE8xvn7WX0i0qn-uHqb7OJSw.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-aSTB2qpLejhHDGKsducYM54rDOdrgJdb-YVdSzSUdkE.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-dWDKup7395XnpxdTYI9wz9tqdLwdbYa-84Z3CIFoQmI.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-8n8TpwRh2rey4WRF5ysUiNBWsBU_2OkvyIQAT1OrPoA.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-FGnJSdgMGJmyiGnBArrO_2QNEWNWWFP5R_MCae5bv8o.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-dTtCv6huFV0d_1PnRyTqLfx-BThtbu374EWeip1FG3A.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-wOBmsEltLpliuqhbtx2JXHUKXgcSc28qGE2TPi4LX6o.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-jgoNSycSx_TLtktkoNctQxEqBQAU6FZPIBMZdL0Kvjw.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-sqVc3iSUvD4qqyRvMhtJKHLDLFQEzqeS7A6AGvmCQcg.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-GWWHq8WhK3bbTpImJ9dGtuPFrXjrmjW-bFiq8x6W0RM.jar
May 28, 2020 3:49:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-_bB3_YB0-Rm8v-QApmuzZ95q5xZ7TvqtN_1FC_ZAlt0.jar
May 28, 2020 3:49:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-xQBwxOKDNEfeML-RuJR6ogM64kJdQzSWEPge6G3Tg4g.jar
May 28, 2020 3:49:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Sd92-Ks1AdYARu83NqxzCIkljgPEtMf83GQG4N1S2QY.jar
May 28, 2020 3:49:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-a3G4BWtHpWCQM_NP2BCcR0PK-wP9E5AaD2CB5hwpKdg.jar
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 1 seconds
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91698 bytes, hash bc6997dd198741c03f302d8a7ff81c75325c87867cabdeb0062540b2d0014192> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vGmX3RmHQcA_MC2Kf_gcdTJch4Z8q96wBiVAstABQZI.pb
May 28, 2020 3:49:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 28, 2020 3:49:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-27_20_49_22-15238145740352696253?project=apache-beam-testing
May 28, 2020 3:49:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-27_20_49_22-15238145740352696253
May 28, 2020 3:49:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-27_20_49_22-15238145740352696253
May 28, 2020 3:49:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T03:49:22.040Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.105Z: Worker configuration: n1-standard-1 in us-central1-a.
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.757Z: Expanding CoGroupByKey operations into optimizable parts.
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.799Z: Expanding GroupByKey operations into optimizable parts.
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.840Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.934Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:30.971Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 28, 2020 3:49:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:31.007Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 28, 2020 3:49:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:31.042Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 28, 2020 3:49:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:31.424Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 3:49:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:31.510Z: Starting 5 workers in us-central1-a...
May 28, 2020 3:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:55.656Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 28, 2020 3:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:49:55.693Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 28, 2020 3:50:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:01.084Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 28, 2020 3:50:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-28T03:50:03.456Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 28, 2020 3:50:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:19.170Z: Workers have started successfully.
May 28, 2020 3:50:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:19.209Z: Workers have started successfully.
May 28, 2020 3:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:51.175Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 28, 2020 3:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:51.494Z: Cleaning up.
May 28, 2020 3:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:50:51.593Z: Stopping worker pool...
May 28, 2020 3:51:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:51:35.472Z: Autoscaling: Resized worker pool from 5 to 0.
May 28, 2020 3:51:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-28T03:51:35.530Z: Worker pool stopped.
May 28, 2020 3:51:40 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-27_20_49_22-15238145740352696253 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 97180468-1c98-420b-8de5-057473d678dc and timestamp: 2020-05-28T03:51:40.733000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.716
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 28, 2020 3:51:41 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 2 mins 33.016 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 44s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/zgztodvggdyt2
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/555/display/redirect?page=changes>
Changes:
[heejong] [BEAM-10077] using filename + hash instead of UUID for staging name
[lcwik] [BEAM-10016, BEAM-10094] Disable single test to get portable validates
[kamil.wasilewski] [BEAM-9936] Reduce the code duplication across python/container/py*
[kamil.wasilewski] [BEAM-9936] Create SDK harness containers with Python3.8
[iemejia] [BEAM-7770] Make ReplicaInfo part of Read on SolrIO
[iemejia] [BEAM-7770] Add ReadAll transform for SolrIO
[piotr.szuberski] [BEAM-10102] Fix query in pubsub read grafana dashoard
[kcweaver] [BEAM-10108] Update Flink versions in publish_docker_images.sh.
[github] Add a flag to tune the size of the grouping tables used by the dataflow
[github] Add Pub/Sub Lite to in progress IOs
------------------------------------------
[...truncated 295.35 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 27, 2020 9:08:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 27, 2020 9:08:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-PzxINxT8sAzka-4qvNs_25XZ9t1v9LH48SfFrl0ylQo.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-C8xt5lToNKO1ctE1P9C1nrAFvdQvjkXGUlae3tWKs-g.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-BYWr772hEM6WjA5BoRNvuuFvZDihPqTKOazJxVy5WIg.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-KgwTnmub75nGiS4UY1Bg6CSxViMwE7I8oFscOGZDnV4.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-8FTPPas8QaYpdNQMlViMVY_rXypXNTKjWedVRdLTg2Q.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-wiEJ-pgrFLxofCiArqiCtehci6Xpiczj6iWyC4BC2GI.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-eP1QzMZ1fvpjRFXCN6fGQWnSsechTPO2vvqgP5XQbss.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-Hi0mUSGR3UbPKbeg16w0DiO5lJ3eSlz3ev3x9mujfGc.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-uvwA4kHvsVcc1NZW8WSHO_MoM0yg5rqS3XaJl0jUv68.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-47GyVf3AmiVRujBD7CvfgEFqZFy3nihaPue2n93zb4k.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-vz3tng3xjO-3N5VqqQn6FJ3r5I_ElGjTHiwMjE_oR0w.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-eRtLQcsmDdnl1JLyLB1ORxz2M1Zu5NyHzeqW1dh150w.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-_oAnfO5Y4QBhLJzIxz7PDtWJPmuUuaIp8mz0f7Czovc.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-__n5HOoLKHfCyL_w_szJq2CiWCOQEG189DSqqF0tGGc.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-3P8HcSQz1JWYE41ILsZHMUT3nndK6ObPjW_k1E7SRtk.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-JIjnfS_D6ScTtTLym2289-ebMqmI1IuvOUjJ8JwWe9I.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3169708779483699546.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-kNDK7JO2Jw6gCuNaQ_WFuz83bclOop3D2I85tNS8OQ8.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-6DfcJ1OHNXMXRFfXMYpTcnBHsSGnF-D9zUUjFHVoo78.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-HGg_fbQfaPKu97VOtD8zIMmEvF0lbdoZo5GdISNSE_Y.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-h6sBwvXSjdh46zUSKQcWcO0sDe74utxuIFsPXxbLB7U.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-LIhDSeDHXQ1HFgk4DN0f6gEam_4yhTayVWihKUIpVuk.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-BAc1zy5N7B0lLJf5v5DIlAh-IC7Sw9pWy8qMYnzpdSc.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-q__9obahXMKb3b-T8sNCafbXz_x_qZYUwU2C2j8Vtuk.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-BB-1dvRtJCdR21_zax38b0pW6Nlku7MGOem33wbb3sU.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-zBbYE23WVdYYzbiCBJbCCjydaPYxTJ_saMhTuEYlAuA.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-rUNZ8zWI3I3Xy_ZU_C4EdBcJSJk1gPgymuWa2TXznVY.jar
May 27, 2020 9:08:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-47BJ4WWbiYrYzjCLBPSX-FfD_QeJ-HQlkuMh1c2XPyo.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-sdBQE59PdP-aV1gFIXKgDGgz_iGKQ4X8Rg6xtIOKS4U.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.process/2.1.2/986b38302fa10018d5baccee7f655c31ee9afe5b/de.flapdoodle.embed.process-2.1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.process-2.1.2-OasY7D5KRAimcZcWcjFwgi8Qb4B-iff1FfrVeNSih6A.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.mongo/2.2.0/781d14f4e3d9eeb0b4c3e00a4ec165a04b3dc5c4/de.flapdoodle.embed.mongo-2.2.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.mongo-2.2.0-vNy3lJC0jW9u4Cy1AHsqSbjRUqOTX9ycpEmHkht7vvk.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-QiCF_HgtfQpRmr-pDLFroBv4lWDbmnI2-l6n4s3Kdxg.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Wchn1AsLaZX7Yon7acfP0IS34r-XU9QLmtPgc0wjbgc.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/janino-3.0.11-kje3HSMpGA5ZIQ6aqhAO4xNFTvCuWIYIx1yxkxlZG-E.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fastjson-1.2.49-1jBOExlmGd1D1lkZf6-tMDRpahbnyycq5E_iuDQCfag.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-compiler-3.0.11-DxpPXyZccBoxkzJErnBF_O8YtPpZUEF-Je5wvlDd2s8.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-calcite-1_20_0-0.1-1NrX_9FNKiEqNk5qBOaRlj-IwqOvKvQIGIbTVgm_v8Y.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-csv-1.8-qL1WZS7UZo2dWjOZSuUvWbnjnI6w68tmhOaK7udXmmE.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-runtime-4.7-KmGUP4A7vR0OAt_9GbkqQY-DNAyZQ0aAnjtR4iMapsA.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-4.7-eGclcCizNzrwEd7nts6bWHqP1cegsl9osv9MuQvoqgc.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/mongo-java-driver-3.9.1-mxKxkvmYluxV-Hdn57uyt-MjjSQUsFjxFw9tjhx0bm4.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/org.abego.treelayout.core-1.0.3--l4xOVw5wufUasoPgfcgYJMWB7L6Qb02A46yy2-5MyY.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/icu4j-58.2-lT4eg7K-fD6i-I2obBNhT0fp5x01eMhSHX8Yd1a2OWI.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-platform-4.0.0-B21i7Yfna9yzdQ_-gKpJXuIRK9chJmOVTWVH9IJEkuk.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr-runtime-3.5.2-zj_I7LEPOemjzdy7LONQ0nLZzT0LHhjm_nPDuTichzQ.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/javax.json-1.0.4-Dh3sQKHt6WWUElHtqWiu7gUsxPUDeLwxbMSOgVm9vrQ.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ST4-4.0.8-WMqrxAyfdLC1mT_YaOD2SlDAdZCU5qJRqq-tmO38ejs.jar
May 27, 2020 9:08:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-4.0.0-2sJwtkQc4k2TqW3bbo-T2N8JkZJzh5mm9vz8KyQWyhk.jar
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 163 files cached, 47 files newly uploaded in 3 seconds
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91698 bytes, hash 06e8b72f331153460215dc57a0e5f1d4523343b0f7d4be805e66f799358f24d8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Bui3LzMRU0YCFdxXoOXx1FIzQ7D31L6AXmb3mTWPJNg.pb
May 27, 2020 9:08:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 27, 2020 9:08:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-27_14_08_37-1571448846652811438?project=apache-beam-testing
May 27, 2020 9:08:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-27_14_08_37-1571448846652811438
May 27, 2020 9:08:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-27_14_08_37-1571448846652811438
May 27, 2020 9:08:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T21:08:37.056Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 27, 2020 9:08:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:44.449Z: Worker configuration: n1-standard-1 in us-central1-a.
May 27, 2020 9:08:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.186Z: Expanding CoGroupByKey operations into optimizable parts.
May 27, 2020 9:08:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.239Z: Expanding GroupByKey operations into optimizable parts.
May 27, 2020 9:08:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.290Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.401Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.436Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.480Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.517Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.917Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 9:08:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:08:45.994Z: Starting 5 workers in us-central1-a...
May 27, 2020 9:08:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T21:08:55.322Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 27, 2020 9:09:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:09:15.619Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 27, 2020 9:09:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:09:30.607Z: Workers have started successfully.
May 27, 2020 9:09:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:09:30.643Z: Workers have started successfully.
May 27, 2020 9:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:10:02.970Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 9:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:10:03.204Z: Cleaning up.
May 27, 2020 9:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:10:03.307Z: Stopping worker pool...
May 27, 2020 9:10:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:10:46.185Z: Autoscaling: Resized worker pool from 5 to 0.
May 27, 2020 9:10:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T21:10:46.225Z: Worker pool stopped.
May 27, 2020 9:10:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-27_14_08_37-1571448846652811438 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 86ae5cb1-82db-4b6e-8edc-a4d91638c8be and timestamp: 2020-05-27T21:10:54.350000000Z:
Metric: Value:
read_time 13.955
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 27, 2020 9:10:54 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 2 mins 34.343 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 1s
104 actionable tasks: 74 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/5yljbe6mytzea
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/554/display/redirect>
Changes:
------------------------------------------
[...truncated 347.99 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8d9e8977-6cdc-4ad0-bb1b-a7f654299e79.jar
May 27, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0e28ba28-f5d2-4ccb-b74d-a29ef6ab1a81.jar
May 27, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e65206b9-0dc5-423b-87a1-6ce40b56866d.jar
May 27, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38b1bf89-1f57-423b-b1f0-2160ca8d293c.jar
May 27, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1fbc16d-cf2d-4651-9e24-fb71bcc16c65.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3252983b-c054-414b-a1ae-f60195ee07dd.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9977be7a-440b-44db-9eb8-c7d5a13ff654.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3fa40348-ed32-47b9-8ead-f21df99a752c.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b38e8122-cb3c-4c0e-bc6f-8c9d387be360.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/64f0a4c9-46c9-4fce-8c31-3875b554ba6c.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74d1ceb9-78dc-422a-8195-6fd2d00ed068.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9c002d14-34f7-4422-9a21-f2961abcdb85.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5b4ac6fe-73c8-4637-8e3f-ff0741bc4773.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2caec504-6a04-41d0-b910-dc5a97c82482.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bc89ee55-6a94-4920-9734-18a47c83b82e.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ede58e34-2cc9-4d6c-86a9-309fbbb73402.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f04e3f95-ae34-41b0-ad1a-1e3064eeb492.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/99c4b10c-6ac9-4f77-9c27-4f2f8ca31b2c.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ec6c2a72-3ff7-41d9-871d-d952c0894e7a.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2262e280-f0a9-4273-99ef-f2aff3390e2d.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/158b4ba4-a66f-4fc7-adc0-5a12e0f61860.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4dbb5a21-ea4e-4600-91fc-16b976542fa6.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7aadb093-c47c-4d77-941b-32473c444ade.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a730347-51cd-4688-a1c2-88aa8f07a4a0.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0f887edc-9a49-4343-bd66-56748dce5bdc.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ce476bf-5ba1-4a3e-bd6b-40713a765c68.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/db80906b-b779-4133-96ca-6d64175bdcdf.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1934c928-013f-4d61-a988-75dac1519eb2.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6833894f-a7d0-4c5a-ab14-3dcc846cd2a1.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a16cb9ed-e39d-423a-94c2-0d558fab4057.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f852eea9-b510-4956-8167-7051af1d1742.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0ac7744-db40-4453-842f-28c279b14f51.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f4f26f2f-d741-4220-9192-6aab2a0af04a.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/94545c1e-e3a6-44be-93ab-c2c287bf15fa.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/771db652-5620-492a-901f-2c49b1c93900.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bfe708a-d464-4132-a5b1-957c091c18b6.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b7ae9ae-d93d-46ff-b843-f05f466bfc86.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d687af7-9657-4b17-82ba-18d83e55a380.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1b6c1fbd-2712-4a75-b29d-d3c7c109fb61.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3e408fb9-636b-44c3-825b-d3ab4d2bd517.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b29d4819-3913-49f9-9cfe-4af709cdad5d.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08209fd0-addd-4f8c-b023-1e66fe79af9a.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c85d0be4-d06c-4a99-8e29-88d37b63d394.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70f23acb-aed0-41ae-98cc-c49502891ff5.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb943199-1c3f-4171-b404-842fc19d60a1.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f36f938b-04ac-481d-a383-bcc036318801.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/64f67cb0-def4-484f-ada7-30cf1e97afb4.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d2128fba-8239-4d8d-ab69-341f0bcd5815.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f39d86b7-639f-4b84-8925-d3f84b75861e.jar
May 27, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa311f9d-6290-4b7b-9a07-78f3fe891f5b.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c1425fd-e53f-4cf5-a26d-492db6a20778.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe20e71e-ffb1-421b-b89a-975abeb75d87.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/305d63b2-0e5a-4b4e-85a8-7baa7fb94fc2.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/208d4c28-d8e8-4309-918a-da4cdd33389f.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87cf41c2-e0d2-4d2e-895c-123af06c3186.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6a26ff74-bfba-44d6-b566-39912c254a98.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ffc46c33-8adc-4025-bd1e-a72d2e146b4c.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f7ddfe45-5f19-4c04-8d46-0e38ae142854.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a2eb18d8-b068-4a91-8747-b1bbb52332f3.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6f027e3a-479e-4ef0-9e81-1cbf3a3e60f4.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b84b41e-ff32-41a8-90c9-ff7ac98ba19e.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8c3ebb2-5c91-474c-9314-6890eb86a27b.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/447e2e12-3dd4-4daa-af23-56a95e6bb0cc.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/57248fc3-f185-4f33-8f85-088dbbeb4f59.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3bae9b59-50fa-44a8-b479-00b7d17df7f1.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d20f2411-00af-41e5-bf04-9b90cbb040d9.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/01dbe6aa-253d-4f99-8b07-0010d3a1d883.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e727383-43d4-476d-bb92-7f0253b41e85.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b8c49dc-3819-417e-b6a0-2c19a4e3f7db.jar
May 27, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ee4597e-1248-4c48-a7ec-84506e993416.jar
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 6 seconds
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 27, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash cca6d1c3010654bdb631ce87153513180d65167ebcc051c7fcd51af76070c7b6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-59bec755-3ace-4d4d-823b-1272f8a7e1a1.pb
May 27, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 27, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-27_05_45_46-11041700056115228896?project=apache-beam-testing
May 27, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-27_05_45_46-11041700056115228896
May 27, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-27_05_45_46-11041700056115228896
May 27, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T12:45:46.243Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:54.262Z: Worker configuration: n1-standard-1 in us-central1-a.
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:54.864Z: Expanding CoGroupByKey operations into optimizable parts.
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:54.906Z: Expanding GroupByKey operations into optimizable parts.
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.027Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.127Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.164Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.200Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.240Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.653Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:45:55.744Z: Starting 5 workers in us-central1-a...
May 27, 2020 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T12:46:01.312Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 27, 2020 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:46:23.065Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 27, 2020 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:46:42.494Z: Workers have started successfully.
May 27, 2020 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:46:42.531Z: Workers have started successfully.
May 27, 2020 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:47:13.438Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:47:13.636Z: Cleaning up.
May 27, 2020 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:47:13.732Z: Stopping worker pool...
May 27, 2020 12:47:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:47:54.652Z: Autoscaling: Resized worker pool from 5 to 0.
May 27, 2020 12:47:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T12:47:54.713Z: Worker pool stopped.
May 27, 2020 12:48:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-27_05_45_46-11041700056115228896 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1eeb2681-3d3a-4bb6-b359-68bafa19f640 and timestamp: 2020-05-27T12:48:00.523000000Z:
Metric: Value:
read_time 13.533
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 27, 2020 12:48:00 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 2 mins 33.789 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 43s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/t3k5vipquytwo
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/553/display/redirect?page=changes>
Changes:
[rarokni] [BEAM-9770] BigQueryIO Patterns. Add Document update to snippit update.
[github] [BEAM-10076] Fix dataflow worker status page rendering (#11812)
------------------------------------------
[...truncated 347.76 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c409ba78-d003-4308-a0d9-a5dfb4818408.jar
May 27, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b9a6cce-6cc8-4407-b9a0-66f6b598ce82.jar
May 27, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44844395-11b5-44ae-95d0-0c9e956a357c.jar
May 27, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f5404e0b-75bf-4aa9-9eeb-4030cb0ef737.jar
May 27, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/77b46597-be23-4541-b3d9-83c11fa12cb3.jar
May 27, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5e03ec1f-31a2-4704-ac27-3442a3aa04b6.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73042bce-0a1a-48ac-a524-062fe95467da.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a4f57e0-eb34-4018-b0c0-c0a7c2e3f2cb.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0f722c5a-bbf1-41ca-91ce-c56bbfa50c00.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ea71e75-7e81-4bab-9f70-b9af62f6e494.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4250037f-d902-4d84-8c2a-d1c3952169c7.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ca57e483-7f4f-4bdb-93cb-0f18e6c555f7.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/522e62ec-fc63-4883-a9b5-e6238ee2b17e.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2d487bf-a3d8-432a-8698-98f4e931c089.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/011f307e-50d3-4a77-ad6f-35e542395afe.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a4c66386-9c78-4831-8d31-06a04ccdc6c2.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00220324-831f-4ef5-aa1b-a2af22e49066.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0f0b863-7e8e-4325-944e-726651f7d575.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a0c71835-ee2c-47ae-81a5-73398a6f4ece.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/30ed91a8-8cd7-4788-baa7-fbacb9694678.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a574580-0b05-43c7-abf7-36914cb74417.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a759b1ba-471d-4c14-9be3-4805f23ad3dd.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5c37ec0-0198-4c3e-b7b4-849edaaf1c38.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1254976b-9508-45d1-823f-08d23f26f983.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed8e0978-385c-48e2-9519-9504412ac84b.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/48fdfa24-284b-46eb-a812-40f2ee17ebc6.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/916f75b8-e144-40ed-b9af-3644fe173287.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ec23451-0895-471d-9ed9-f8f40a8dc1fa.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/122d48aa-c958-4e45-87f6-d3bb2c64c7f6.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9e445f04-fc76-4acf-8d05-ef80b71d08e6.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8c7b400b-c215-4e1a-ba22-e4990585dc85.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85cb3713-90ee-4259-90db-50ec19c925cf.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/727541ad-d4d6-4fef-bde0-988c89df0b29.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/995d8971-6dda-4257-8071-8311536b5110.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d98f1d45-61b9-4c04-92df-b4b1ff51adfb.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/96a010ed-704e-4cec-bfa7-8a39672c2769.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d87ac427-b9f3-47e4-b8d8-b65e17d86285.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/99c46fc8-4e43-4418-a7b3-03cf29b8819c.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/60b64e18-9704-4433-b680-1a99222e62e8.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba16dc92-d4e8-41f6-b0b5-8bcd70273ac1.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23fcb398-39ad-4927-81be-dde1489b9356.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f87e1709-76c2-4da5-91d5-7836cb85b861.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c9742b1-5ba8-406c-bac1-48900ec20138.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/68c2fdc0-c2f6-42b2-8c15-fc24cea193c9.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fcce4dee-07cc-4308-b395-11a21fac6f5d.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74adcc33-c956-4142-b9c1-f588788c79eb.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f89edd48-6663-4b52-b46e-7897b1004db4.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a7cc5609-e839-411d-8d01-56662ee8290f.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cbfb6d81-53be-47cf-89a1-fd3b177b6184.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c28b9344-d59b-4d36-84b4-308dada3d070.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/42984759-644c-4590-b4ba-10872a9c0404.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cafce28d-c63c-4c6a-8835-66d61f6646aa.jar
May 27, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00bcf688-dbdc-441b-ae8c-6d0941e29da5.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9557a36c-d224-41ce-afee-2b05857c3aad.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1b769a61-508b-4844-a135-b2fe70687200.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bed44d99-8c2b-4aa4-a2d9-3318a386b415.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/085ce21d-dd78-400b-8dfa-f656c4ebd7b9.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e22aa9de-9e99-4e02-98f2-8461222139c5.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/05924e8a-ff91-4bc9-9e0f-6a530b438d2a.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f26b940c-42db-49a1-b2db-061c4f0a8be3.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00cbeafb-fb18-4782-b4e4-5f71cd0086a6.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/056e7a42-85df-4a8f-a72c-df870a9e2865.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0991d664-e4bf-4101-8e01-b8db2ec3d2a9.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74a11c0e-292b-4352-bd8e-0e0ee7fcfed7.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a28354ef-efe5-49fc-b213-110f6df0edbd.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d928d3ad-ae51-4dc0-8684-64d4284f4880.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/12f907ce-6512-49c3-a5c8-04c628cfe404.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bff14376-46af-4b84-bb6b-7e837a31e611.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ac1f4e1-a786-4f10-9dff-fb14aeeb6122.jar
May 27, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2b3055d0-79ff-40ee-a690-dd533fcde9fc.jar
May 27, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 27, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84252 bytes, hash 48f28b86acf19dfcbaa9b496235dd2f6ab4c1db0feebe5ec2bc8f59b824c6638> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-c97a4e3b-6209-4dd1-81f7-881380619444.pb
May 27, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 27, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-26_23_45_50-13921750635286393215?project=apache-beam-testing
May 27, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-26_23_45_50-13921750635286393215
May 27, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-26_23_45_50-13921750635286393215
May 27, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T06:45:50.727Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 27, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:58.533Z: Worker configuration: n1-standard-1 in us-central1-a.
May 27, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.168Z: Expanding CoGroupByKey operations into optimizable parts.
May 27, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.212Z: Expanding GroupByKey operations into optimizable parts.
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.254Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.334Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.373Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.417Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.451Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.857Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:45:59.951Z: Starting 5 workers in us-central1-a...
May 27, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T06:46:08.569Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 27, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:46:26.898Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 27, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:46:43.721Z: Workers have started successfully.
May 27, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:46:43.755Z: Workers have started successfully.
May 27, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:47:18.561Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:47:18.795Z: Cleaning up.
May 27, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:47:18.889Z: Stopping worker pool...
May 27, 2020 6:48:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:48:09.359Z: Autoscaling: Resized worker pool from 5 to 0.
May 27, 2020 6:48:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T06:48:09.419Z: Worker pool stopped.
May 27, 2020 6:48:16 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-26_23_45_50-13921750635286393215 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): eba74ee9-d331-4b8a-9a3f-2f501b82456d and timestamp: 2020-05-27T06:48:16.529000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.862
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 27, 2020 6:48:16 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 2 mins 43.854 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 57s
104 actionable tasks: 69 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/riryvohpa5hmo
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/552/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-9603] Add timer family support to FnApiDoFnRunner
[zyichi] Address comments and add tests
[zyichi] comments
[aramaswamis] Fix typo copyLicenseScrips -> copyLicenseScripts
[aramaswamis] add io-matrix
[aramaswamis] add descriptions, links to guides
[aramaswamis] fix typo
[aramaswamis] format
[aramaswamis] update CHANGES.md
[github] [BEAM-7370] Upgrade sphinx to 3.0.3 (#11798)
[zyichi] Clear redundant debug statements in pr-11756
------------------------------------------
[...truncated 348.05 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88838334-2622-4dbd-b4bd-09215cedad32.jar
May 27, 2020 1:18:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.14/3cb1181b2141a7e752f5bdc998b7ef1849f726cf/commons-codec-1.14.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3fca2655-6600-4c04-b173-465fa5295a8c.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/94e24cf3-1212-4c15-8da9-876deec87ba2.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f5fe58d6-7d2f-44ad-ad3d-2dd3e273f87b.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7da1cb76-89dd-49f7-b39d-91b8ac0f2516.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dcfc17f6-c345-445f-82e9-51c5c814282b.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/19f03080-f11d-4c7a-8dd4-68cb4394a13a.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7ee2228c-89fb-455e-bd17-2509024b10be.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e45cca71-e6c2-4a45-be3f-3a2e403eae61.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0605cd7d-fcad-4c10-9e92-f727b276f8f7.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/626640c6-1a22-4c89-99a0-57eb3e0559c6.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85d95a2d-2b90-4c83-8ecf-124a19efd187.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/378a12b1-89c9-46d1-8047-16d0a992f127.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9bf0f94-7a90-49b1-a28e-2b6bc4f3f880.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c09ccb71-3125-4e3d-bd3e-8e0da93c2307.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45975c20-350e-48fa-8e76-88402243b5fc.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e9f95e3a-63c6-47ff-9487-1c41dbca4125.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ad5f91d3-2261-48f2-97c9-92f835b78a37.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/78da545d-0058-4536-8dc7-f08b80798184.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/badd54cb-9769-45e9-b8a6-36bf6324467c.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14263cf1-d76b-42f5-8f68-4ef8e841bf1d.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3176065f-9144-4bca-b40b-44cccfd68706.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/245adb83-5483-4f7b-9aa6-47a242b0c731.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fbe215ea-f3e3-4a45-b34c-464b159c0d2c.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5f10e020-4e89-4206-a60e-5c53b39e01e8.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/da1be155-d344-4b45-8b72-44bd7dc94b63.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7ce2fff9-462a-41ea-8110-00865853551b.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/be5c53bb-10fe-44e0-82ab-dd24622de130.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/420112be-c0f4-4777-909a-c26bfcf35049.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27400475-c534-4766-88bd-8e887ffd52d8.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed71f67a-c839-4937-b0cd-50c6652bf105.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2db79dc-857f-4285-a7df-6f1f74c3ddda.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd701532-a163-4577-a379-e31eba379907.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1e83433-a7d7-4605-be58-fb2177cd0837.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e5a7f9ca-dacd-4e84-8472-9aafeb90e853.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a4838f02-1d82-4da5-8dea-645cf8f1afe7.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/77645384-4eda-4ae1-8d87-1f54d124e47f.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85ba9092-14a2-4263-afc0-ee047e59941a.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d9d4ac8-a602-463d-b314-c957883aaa5f.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e118322a-dd31-47e7-90d6-4707972b4d22.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/86078a6b-1b13-4065-9d3e-d52925420027.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5fd2373c-4582-493c-82ce-7af59a98b305.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d7f9ee6c-2541-4486-808f-302cc285a862.jar
May 27, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ace0b4e-2412-4f02-9bcc-35cec8bcb449.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/72608a1e-3fd4-400a-a921-6e65bc5753e3.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0ee42277-2239-4910-9d69-65cf18fd51f4.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e5742ab6-84d5-4dab-9568-1a04c7fe88fa.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bb7b09a-20cd-4f0a-9d4f-8933519898f4.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/99ccc83b-a639-4a82-807b-4c9d8bfa3068.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06e8ddfb-4210-4983-877b-e7b59601d1f1.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ebce60f-6b4d-44f5-aa82-a69b06d040b4.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e17d4b3-89be-4dc9-a5d0-63d78b736c93.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/285fd453-1b20-4cad-ab8c-047e3be4afba.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb2e2062-84b1-4e4d-a117-78360cdb8013.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/285063b8-dca3-44e0-9c73-1c3cd52a54ef.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1c780405-fe11-4ce3-a8b2-d14255c0c5dc.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/168b3831-ad87-4cc1-8c4e-c2cbc1adb2be.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5c395b1-c2c9-4118-bed0-db6ffc9fd901.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/514e4490-f747-4f9e-bc08-eb94fa961b76.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c8d4ae1-ac32-4526-93b4-c5a6bbe7f068.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b3aa24a-bea6-45f2-bfa0-3c10ad321733.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45d23176-8c0f-4f8e-9366-a1742007da43.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/34ccbd84-28f5-4a83-bddb-33dcbeb95e2a.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/139e73c2-f606-460c-a62c-b6347be7431e.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee195420-90a9-46aa-929d-2bf2e19c3113.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/adcf8a56-4ed8-4413-8193-8aa6820eedbb.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3a8e100-7c95-4c00-bd25-79511a91ac56.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4548be2a-d178-4cb3-b283-06899f3cd434.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1714be61-bc34-4f6b-b63b-77cf283aee1c.jar
May 27, 2020 1:18:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/905f2fa5-ba4d-4980-8060-36a45929d80d.jar
May 27, 2020 1:18:51 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 7 seconds
May 27, 2020 1:18:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 27, 2020 1:18:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 27, 2020 1:18:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 27, 2020 1:18:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 27, 2020 1:18:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 27, 2020 1:18:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash d373ae8e32180d657669f150c46c0edf0b2ebd7c88838cb306c0fe98eb42f019> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-e9fc5f5f-7e49-474f-8dd5-6c74dd5bd817.pb
May 27, 2020 1:18:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 27, 2020 1:18:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-26_18_18_53-8320908097293167156?project=apache-beam-testing
May 27, 2020 1:18:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-26_18_18_53-8320908097293167156
May 27, 2020 1:18:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-26_18_18_53-8320908097293167156
May 27, 2020 1:18:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T01:18:53.585Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:01.239Z: Worker configuration: n1-standard-1 in us-central1-a.
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:01.914Z: Expanding CoGroupByKey operations into optimizable parts.
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.159Z: Expanding GroupByKey operations into optimizable parts.
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.198Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.313Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.348Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.382Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.413Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 27, 2020 1:19:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.835Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 1:19:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:02.924Z: Starting 5 workers in us-central1-a...
May 27, 2020 1:19:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-27T01:19:18.537Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 27, 2020 1:19:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:29.063Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 27, 2020 1:19:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:52.947Z: Workers have started successfully.
May 27, 2020 1:19:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:19:52.975Z: Workers have started successfully.
May 27, 2020 1:20:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:20:29.392Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 27, 2020 1:20:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:20:29.582Z: Cleaning up.
May 27, 2020 1:20:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:20:29.670Z: Stopping worker pool...
May 27, 2020 1:21:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:21:13.530Z: Autoscaling: Resized worker pool from 5 to 0.
May 27, 2020 1:21:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-27T01:21:13.571Z: Worker pool stopped.
May 27, 2020 1:21:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-26_18_18_53-8320908097293167156 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a08990c8-27d5-4f61-9831-1cf7758a8ccc and timestamp: 2020-05-27T01:21:19.511000000Z:
Metric: Value:
read_time 18.267
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 27, 2020 1:21:19 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 2 mins 46.837 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 59s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/qsliigoaodrac
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/551/display/redirect?page=changes>
Changes:
[bhulette] add def args
[github] Fix typo Portabillity -> Portability
[github] [BEAM-10065] Fix beam release guide template (#11797)
[github] Fix typo in doc "and and" -> "and" (#11810)
[github] [BEAM-2939, BEAM-10057] Ensure that we can process an
------------------------------------------
[...truncated 353.44 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/086bca67-a0ea-4492-acbd-1e9dd7adec6c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e149f5d2-f40d-4023-983d-e9cb2d89b164.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e4e96f8-3d0c-4519-9ca0-cc3f23cef97c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6f540eed-d4a5-4d48-9d26-0b939431d7b7.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11fa03eb-b06f-494b-942a-e820457a3b02.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b81730d3-3b68-42f8-a94f-06cab704fdcc.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/369b2f5c-a4bf-411b-81a2-11921b9551f1.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d5206b2-b1a8-4ef5-b0a1-09f5097ba88c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/238c587a-7fd8-440b-8513-faf478137df7.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e44549d4-c9cd-421a-954c-0e8bc1ab25a6.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e7c29228-a2f2-436c-996e-5e041815700c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5deecd48-e9de-4e20-8269-4c797fc2d6c0.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9706fa7-51d9-47dc-bfa2-45cd5d134ab4.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8dd5451d-50e2-4e20-83a8-0df2c177d13f.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/acb48231-fd9e-420c-97d9-772832c71ef8.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/124dc25f-036f-46f4-9fa1-e323e369b100.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c675362-baef-41b1-bd31-47357cddbad8.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a64d76cf-932c-4b09-b51b-81bd40e1f129.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e61fe379-3792-48e0-8d10-2fc502c02c28.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49dad050-b177-4beb-8a52-620194739175.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eb49789b-19c1-4e5c-b9fb-9512e3ef25e2.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d094591-22ed-46ff-897a-7de33c8ec0bd.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7fd59bbb-b154-4668-9714-0f4ded518e23.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3f537534-a06a-4e43-ab26-00b6aef5aa8b.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4693d168-2829-47e4-8fde-c3f8f08a3e48.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bc8260ee-e530-4fa9-9c48-e0c3d73194ce.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b765244-770c-49a6-962d-e71423b0d48c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3bcffd9f-2cc8-4b83-b7ca-e1076c179417.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/acf4eab9-b120-472e-a903-5ad373cdc0a1.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9f1eeda0-ea3d-4353-9636-505c22d0a907.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/76a33769-6e31-4a63-a2d1-a0c7af27450c.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/71a56025-5494-41fd-b9ce-1e684d44e992.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1efb1861-ecf9-46c5-8dff-afb94c8828ea.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73b5f199-55bc-462d-8dab-e0f01aba94a5.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e12005db-eb7f-4dd0-afed-1f224fc409bd.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2786b407-966a-4170-bcf3-bcc46d219c4f.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd08bd33-7352-4f58-ae12-c3d4bf1483f2.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6aa6ba96-3f27-46c1-9385-e76c26c38842.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7cf6dbb5-7afe-4c4e-8e64-803d6fef5606.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4ef70769-1e14-4e9b-995e-0721bc29f444.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ad1678bb-2cc2-464b-b5ae-4450a4346442.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/da3fcae1-b335-420d-a871-c68628041e02.jar
May 26, 2020 9:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef1f5f03-7dd4-49e1-8749-1dec545b91d0.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f8eea6e-0fc5-4216-8ef2-28a2858c1a37.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fb79a4f7-2a5e-493a-aaee-54348c0717b6.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/34867492-e817-4d9f-8181-86146b65394e.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7111c77e-968b-4e44-a808-7980f792c4cd.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f00a42d9-b055-40d5-811a-218dbf2ba497.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/014217da-1fb7-412f-85e5-5d2259fd418a.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c98939e-fbd3-4260-9b34-01f0a9644d7f.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25c7fd14-cf8d-4cdc-8cfb-d444ccd3ca3d.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c86226e0-91f6-4f0a-91f9-ec7e6405c091.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a03b8d27-8e70-4786-915c-5cc2db2cd0ad.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3ebc263-38d6-4011-9db8-bd2eb3c8b4b5.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3c9fc07-8764-417f-977b-13f1fac3b4ea.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a703fd97-a101-4e19-a52c-51985352bd71.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8805b880-12aa-4e56-b922-9c3f86b86207.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/80ed7067-8bba-413a-99d7-f21440e4f9c6.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/042a01f4-0888-4853-b418-65ffa02ae95c.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bcdad1b9-665f-4704-a824-1355423b5683.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8469cf01-95e9-409e-a37d-e4e4c5857703.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6fcc9b6d-cabe-4140-89c6-a3fb41d8f7f3.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45b6d189-c0c5-438f-9885-917e03cf1478.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/306a4783-352b-4a53-a455-e80ba83b87fd.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f98d83b3-838b-45a0-a241-1a504bff885a.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b4aa513f-fbe3-4fa0-9ee8-f1f536b760bf.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/1f9cadf5-1237-43e3-86fd-d8081f8bff32.jar
May 26, 2020 9:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/4f681c30-bb67-4973-9aa0-f34be3056033.jar
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 7 seconds
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 26, 2020 9:01:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 26, 2020 9:01:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 75f8aaca400bd6cde9a3739efab17052a0fbb184ed79179f6e573a8453930868> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-f8a7a659-9504-438f-a625-bc3b39f43d9d.pb
May 26, 2020 9:01:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 26, 2020 9:01:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-26_14_01_16-13257719985302097369?project=apache-beam-testing
May 26, 2020 9:01:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-26_14_01_16-13257719985302097369
May 26, 2020 9:01:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-26_14_01_16-13257719985302097369
May 26, 2020 9:01:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T21:01:16.908Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 26, 2020 9:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:24.358Z: Worker configuration: n1-standard-1 in us-central1-a.
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.212Z: Expanding CoGroupByKey operations into optimizable parts.
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.260Z: Expanding GroupByKey operations into optimizable parts.
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.288Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.359Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.404Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.446Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:25.481Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:26.003Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 9:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:26.109Z: Starting 5 workers in us-central1-a...
May 26, 2020 9:01:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T21:01:33.192Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 26, 2020 9:01:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:53.148Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 26, 2020 9:01:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:53.185Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 26, 2020 9:01:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:01:58.557Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 26, 2020 9:02:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:02:12.694Z: Workers have started successfully.
May 26, 2020 9:02:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:02:12.727Z: Workers have started successfully.
May 26, 2020 9:02:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:02:44.128Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 9:02:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:02:44.364Z: Cleaning up.
May 26, 2020 9:02:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:02:44.446Z: Stopping worker pool...
May 26, 2020 9:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:03:46.481Z: Autoscaling: Resized worker pool from 5 to 0.
May 26, 2020 9:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T21:03:46.531Z: Worker pool stopped.
May 26, 2020 9:03:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-26_14_01_16-13257719985302097369 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ed616410-7178-40a2-88e3-de33b57ebde1 and timestamp: 2020-05-26T21:03:53.168000000Z:
Metric: Value:
read_time 14.632
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 26, 2020 9:03:53 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 4 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 2 mins 57.721 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 13s
104 actionable tasks: 75 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/htf3fhlnvb46i
Stopped 3 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/550/display/redirect?page=changes>
Changes:
[github] [BEAM-10073] Add pubsub performance tests results to the grafana
------------------------------------------
[...truncated 347.17 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7cfba2a6-3685-47e8-a9b7-10bf1f0bf574.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/572716d4-ccee-4a4f-aa70-1e0560c8d575.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7c67c355-bdda-4992-9442-b3b419970805.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2b269b98-81fb-456c-b8c2-a096f538d1c2.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/21099bd7-b5da-416f-8bf7-1aa99d21fd8f.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44aa40d8-a201-4462-add7-d012f7221204.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/97e4bab2-6737-4e05-95d4-d96d66c87bcd.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1811eaf-dcb5-4583-9235-ce6005f0b70b.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25c97034-47f7-4c1f-8e8a-cafc2cfb0c5d.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c2c1f27-5d6a-429f-865e-e2c744b0add6.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/51449f55-90ca-4b98-b755-f7fcb817c040.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe026ee7-9cf5-4890-b9df-5f2d5f47b3b4.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3631b377-337e-44a9-aa7d-70047e8e9bd0.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0135820c-558d-4339-8a5c-dc8a5fcc2a82.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/600c899e-32af-4115-a636-ef1c1c3992a6.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8d0367a4-00f3-4294-bd3e-72ace29d1745.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33424dff-412c-4a30-88a2-0b4bc5bd6708.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef3bbe40-9b74-4b5f-9e5b-8c0e6fce99b7.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ebcc455-e89c-44c8-b5a3-916798fd2147.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a4611e2-9029-426d-abc6-cd8adbc869b1.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61378610-b281-471c-9f45-7734be3f8c36.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1339c3ca-21f6-4950-a6d0-d395e443cf5f.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/69930371-e8e1-4e28-a3f2-89de339c6d55.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16d26bb5-4046-4981-809f-bcb05ae68ab1.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c94603f6-0e26-47da-a5ed-601f1db88277.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38582ba0-91af-4f6a-9a6d-1be34c5b2e66.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33d7cbb9-44b1-4c08-b2af-1e01170e8f24.jar
May 26, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/319730f3-f004-4cb7-be2e-9fdbb2d8df8a.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7c098ebe-368a-40e2-9b0f-20f4cf9734e0.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2c5d2e56-a7bd-4050-bc50-07e5e3d1032b.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f32b0b87-abef-44d4-924c-49f9bfda2486.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a69122ff-4262-447c-a798-3653cafbed7f.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31159a51-97b5-4702-85eb-23cfa6ef4cb1.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88d82866-2486-4753-a91c-551a58079804.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8cdcfe60-cb09-42da-aae5-3de3fe0294fb.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b7a644f7-555b-446f-b2b5-885321b9d938.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9df42e2a-e913-4a50-bb67-15d159af70f5.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aad7b838-52a5-4ce6-aeca-7e45244ee6f6.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa14c5cc-d6ab-4794-b2aa-422f286e507d.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a331f486-32c3-45ba-8e42-875c8f3a9b31.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/062b71d6-db8a-4d08-905b-1d4975f98eb0.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/89716b2c-a888-49b5-8402-81015ad344d7.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25b0ba4c-6efa-4e87-9fa7-d5dfe1055eb2.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b2b7fe75-9a7f-407a-befb-7e2558334527.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1ec9064-c7f9-462a-a8be-8cb9fd8bd26c.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3fdc5f0d-7a68-4107-860f-2722c474a364.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f1899cce-8dd2-4dd2-915b-5e91f996e5a5.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a934400e-d1a0-4c0d-abd5-f2cb9628a2ec.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a5d1701-65c1-4d85-b0aa-3c4e994a4d4c.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cca20024-a344-42fd-9ba2-913e200a00ff.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/03ae6f7a-99f2-41d5-afe9-3a4277099841.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1f81bfa7-ce02-429d-af02-6d8dc7aee0b0.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1a6200d-5360-41eb-bb05-a11a618c9674.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e618c718-7f5b-454c-97a4-fe54b1ef121d.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba75da30-b598-46b6-8760-8a63a844d482.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/09fe85db-e804-4d1e-a4bc-37a081bf8942.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a2872df-5ed4-47ee-94fc-bcffc789f646.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/680407d2-8104-41d1-a4c6-5b2f47b01376.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/867bf287-ca1c-4c26-8249-f6d9ef07d1b8.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef9bec95-1566-4fda-927c-02e3c01ba3bb.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/529c1c52-379e-4b51-bcc3-b69ee38594aa.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a2aea11-c298-4321-a569-3b66ecf5db2c.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0dbf7f96-b80c-4675-981c-f1695e2d680f.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/255387d9-834b-4057-8707-2e68434122d5.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1bdaf96-2b25-47ce-b898-de6ec9bcd409.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/93671185-9bbe-4885-9551-7a8b5e94c1c7.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27af5365-7e6d-4c3b-9276-8ee280d6aa5d.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6eda9a5a-4b2d-48f0-adf8-feb6e0b87def.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/deb02df8-0a87-41e2-9bd7-d43895bfda24.jar
May 26, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/387d3fb2-4614-48aa-8784-3493a5f9751f.jar
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 26, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 34765e1d09e0bc29075d23083ee9268791ba8d69805955deae1d5985e7ac0f7a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1e643e15-6c1b-4a4e-bd88-394996eceac7.pb
May 26, 2020 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 26, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-26_05_45_42-13982636098917155001?project=apache-beam-testing
May 26, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-26_05_45_42-13982636098917155001
May 26, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-26_05_45_42-13982636098917155001
May 26, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T12:45:42.430Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 26, 2020 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:50.414Z: Worker configuration: n1-standard-1 in us-central1-a.
May 26, 2020 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.049Z: Expanding CoGroupByKey operations into optimizable parts.
May 26, 2020 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.081Z: Expanding GroupByKey operations into optimizable parts.
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.118Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.201Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.236Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.266Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.302Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.720Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:45:51.799Z: Starting 5 workers in us-central1-a...
May 26, 2020 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T12:46:09.404Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 26, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:46:15.281Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 26, 2020 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:46:35.314Z: Workers have started successfully.
May 26, 2020 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:46:35.347Z: Workers have started successfully.
May 26, 2020 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:47:08.957Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:47:09.161Z: Cleaning up.
May 26, 2020 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:47:09.258Z: Stopping worker pool...
May 26, 2020 12:47:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:47:54.747Z: Autoscaling: Resized worker pool from 5 to 0.
May 26, 2020 12:47:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T12:47:54.784Z: Worker pool stopped.
May 26, 2020 12:47:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-26_05_45_42-13982636098917155001 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9ed684bd-526a-49d9-93bb-d08c6cc187ad and timestamp: 2020-05-26T12:48:00.015000000Z:
Metric: Value:
read_time 15.327
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 26, 2020 12:48:00 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 2 mins 35.731 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 41s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ofpabeliyfime
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/549/display/redirect>
Changes:
------------------------------------------
[...truncated 346.14 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aab12b1c-fdf9-4f61-a69b-3fbd2066629a.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ed509e6-4d45-424a-83f2-1d3e2aae7b34.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-module-junit4/2.0.2/5b29faa1283332f1368cfa87d688e47ae75b7368/powermock-module-junit4-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3bcc04fc-99f4-484b-855d-bfb4320672d5.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ecf10b6-b778-4354-9466-44488fecd0cd.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38b9f706-ff2a-4f31-94e1-82d8ef5ba726.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9cd8b384-77a2-49fb-b1d6-1fe73821f8d1.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07337966-e898-4c2a-936d-33ea8e922a5a.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b43eb1ce-54b1-47ac-a129-d7738c88440b.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6b68051d-20ed-4265-9181-7c3db8bda62c.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/878f43f9-c30c-4155-b4fd-d05b580c4c60.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1df02a4-1c5e-4fcc-9172-ea0a7bb9f062.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4817a7e4-5818-49fc-874e-18a4c989a4b1.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f0f77a8d-4abd-484b-a5dd-f4f3fc7c641e.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d4c3d2ef-a1ba-4da0-bc02-ee7f74fef511.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3190758a-4a62-4de1-bf24-1ed8cddd122b.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d92f9eeb-ee0e-4120-9829-18921b3c171f.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b8ed78f9-ffdf-4691-870f-93d7e1fa5fd7.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb648112-8c3a-4574-81c7-4bfc0293d250.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/730f66ee-7b3f-4da6-9018-ca0b1da147bc.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c38910bf-3181-449f-9e29-73a75322f06c.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4083b96b-1eb5-4529-84f3-f46cbd7c1811.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b030b9d7-ead4-4062-abd5-cded67a6eb84.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/63a1cd35-9887-4ce8-a96f-c27f2b0d1b19.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/10f9ea02-4910-441b-9c04-0ed701ed544b.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a6f442a-2751-4297-93e8-5c0b92b4b699.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/71aeac11-6bab-412e-96fe-aa3021a75dfc.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f56da9f5-b512-4b63-82d8-8969da034021.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/02d37cdd-e6fd-4697-9cbb-a3b5ac1f398f.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0e6d072e-925e-4bfa-b26a-10ae2aaf466d.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7eec66dd-fed9-445e-b515-484fa39016f8.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c486b1c5-0ec6-4fc5-af37-dd6375ed1e0a.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df57958d-5cd4-4f87-a239-25af67d859e1.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0eac2f89-a47b-45e0-9d0a-5a6ced7ef443.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/40597207-8b73-40d4-9324-4614e3c3149c.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91e15a67-2eb5-4f94-a68b-9ed20c83e654.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1513decf-ff13-44f7-95ea-56f17a10f836.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/113fd524-26f8-47b9-bb4d-45fb97d4541b.jar
May 26, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/63c5f47b-bb65-4f14-b22f-c656c60d9b25.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa50f7b1-d94d-4c45-8f66-9073adcbc2f0.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04796b36-854b-4258-8406-a7877935a805.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/549a68d9-bca4-4738-b5e7-6f0a247111d8.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e518155e-c623-4bfc-a00a-b898092045ab.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eeed4145-771f-4694-849e-66d20d5d592f.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3379033e-ea2e-495c-94f4-3961674d295c.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/26df6c95-022f-407f-99b2-81d1376295d3.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/db75283f-393a-4870-8dc8-700e40f74b37.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce7d568a-9f5a-48ea-b565-2ca2dba8ee9b.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b8c02dc-6281-4948-a428-dde17180e300.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/af9e636a-bf95-4e81-8098-b65c48d6a9f0.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ea0ce562-76ce-4e75-9443-e2a92746113a.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bc1e94d4-1974-4538-a86a-0dcb8f3fa756.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d450dd00-4f2d-4b1a-ad20-382cccfad356.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cc4d5457-bb0a-47e3-98a6-97bf058bd8ad.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1b31dc4-2cc8-49d6-a908-6463a2cf9e88.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1bf1ab65-e09c-4059-a687-bab6778f6b70.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c8cd85a6-ecd9-4f2f-8f43-b4b7563f1a21.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07e118b7-8b45-4873-8527-2cb9ff472eec.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d58685ce-7bc1-4b1b-8415-6dd29a4c32e5.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5d78c390-6f41-47b7-85a5-b42b791d969f.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1434f6d5-8132-435e-a88d-e283218b39e1.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c3a4dc96-66cc-4e1d-bf98-b9f2fc429d40.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/39c0989f-7486-4912-af69-1a4e3744600e.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ad6afab-5a1a-4c5a-a8e8-d1c3f37817de.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f0fe6d91-28b7-4010-9d0a-aa92249e0631.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07318985-6380-436f-8aaa-009c92b55d74.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0bf40dd-ba42-4da8-8471-d3e6fd7561b4.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beb67245-9537-4a8a-82d9-d864d64afdc9.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1545c0d9-1ec1-4d9a-b0ce-e5f72ebf7944.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eaa03f86-aebb-4431-9271-dfc07501f101.jar
May 26, 2020 6:47:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a4c0aea2-389d-4fa2-b35c-0201e5279ee1.jar
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 17 seconds
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 26, 2020 6:47:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 26, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 89ecc9f6ec9241ecc3e1c9d51fa869bb62a268822835630927bf02b00e9cb32c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0bd0713f-819d-4543-ad0a-e38179c758b3.pb
May 26, 2020 6:47:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 26, 2020 6:47:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-25_23_47_20-17583497135019959952?project=apache-beam-testing
May 26, 2020 6:47:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-25_23_47_20-17583497135019959952
May 26, 2020 6:47:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-25_23_47_20-17583497135019959952
May 26, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T06:47:20.673Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 26, 2020 6:47:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:28.282Z: Worker configuration: n1-standard-1 in us-central1-a.
May 26, 2020 6:47:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:28.923Z: Expanding CoGroupByKey operations into optimizable parts.
May 26, 2020 6:47:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:28.963Z: Expanding GroupByKey operations into optimizable parts.
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:28.998Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.123Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.154Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.191Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.214Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.582Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:29.659Z: Starting 5 workers in us-central1-a...
May 26, 2020 6:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T06:47:40.526Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 26, 2020 6:47:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:47:55.519Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 26, 2020 6:48:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:48:16.518Z: Workers have started successfully.
May 26, 2020 6:48:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:48:16.550Z: Workers have started successfully.
May 26, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:48:51.395Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:48:51.582Z: Cleaning up.
May 26, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:48:51.668Z: Stopping worker pool...
May 26, 2020 6:49:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:49:43.764Z: Autoscaling: Resized worker pool from 5 to 0.
May 26, 2020 6:49:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T06:49:43.801Z: Worker pool stopped.
May 26, 2020 6:49:50 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-25_23_47_20-17583497135019959952 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a7eb96e5-3d86-4352-bf66-1e88682ca083 and timestamp: 2020-05-26T06:49:50.730000000Z:
Metric: Value:
read_time 17.408
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 26, 2020 6:49:51 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 2 mins 59.713 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 8s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/u3fynr56fwim6
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/548/display/redirect>
Changes:
------------------------------------------
[...truncated 347.72 KB...]
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e7e20d9c-5eeb-424c-b0d0-5606ab21e1c3.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/487006fa-76c1-472a-a13e-cfd30b67f62f.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e459a054-304f-4764-8e7d-c1ccecad6e83.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5aab626f-d4cc-45e5-b31a-d65d9d76ffd9.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b87b931c-8c39-44ce-a8a3-fcfde76f2188.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/784a37e9-c9db-4340-bc58-815b62094b88.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/96715946-cc0b-48e0-bafa-267d1be8bca3.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b193f384-85b9-4d5a-bc96-85870de900f6.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/331ffabf-c973-4e33-a3db-b62c1b83ace5.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bace398-81d6-4600-b8e4-15be8054ca40.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49704c8c-2a30-496e-8d66-14699cf462bb.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/41e2f7c4-c201-4e65-83b9-f1b32f1be606.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/825d76c8-792c-4b50-9f53-4f1d439209fc.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7c7c2b41-02a9-4bbe-bbb3-1d4d836808f4.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/39769ed0-54df-4167-be71-ec22084e4299.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/34b53a43-e5d1-4663-9d24-954e57b1a240.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/29a7fe34-5833-4994-8257-26b0912cbbe6.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/177f2303-f783-492a-8a48-80d877923e12.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d727c08-0b1a-4fcd-ad99-bd7d7f169f96.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/891c5ca5-3648-45a4-93aa-6fd380991d76.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/75c9d3f0-1900-4eae-b9e8-cfa64fd66ad5.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce08104d-437c-4e24-9579-f8aa8f63bb0a.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7367e891-83cd-4417-8857-321610a17a7f.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c41c7127-4fa9-4ac5-baf8-81125726396a.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9509e3f-2039-46c6-a8cc-d6bb41c93efa.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c0e02127-080a-4890-8128-b37f6fcecddc.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4de7bc33-2e32-416f-aefe-d6aa64982542.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f952eb0-4f87-4136-9b60-0110fb74f2f8.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25a45c2d-a485-4754-91ae-ae22b21ecbe7.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/95e8f72b-0d60-42cf-9e82-e4a96c1f3325.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d2f36ade-f412-4a15-9670-73e58b9dbe94.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f3713c9b-1e12-4210-b1e7-6f4dc124e33a.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dbb62ab6-630d-47e2-aded-6c4b0018f148.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/24ad80c4-5d58-475a-bc7d-b37e9f3e4349.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/314ca763-31fe-46e7-bb79-ea30f2e328df.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bfe12be3-d4d0-4660-8390-33bc19a4adb7.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/507dc8f4-5ccf-45d1-a891-2df2508d2a71.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/717692eb-40cb-4e38-9fcd-44f55c079bda.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/13b875cc-7c13-479f-8390-e06ae3f02e5e.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/be5fdd2f-dad7-4c5b-8901-8c2895160469.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3931796b-36d5-4d19-aad3-1c39deb72ec6.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7eaf6681-a323-45e2-ad9d-be17b3f5bee2.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2bec41cb-143b-49d9-b219-618815e8e888.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/51fe6475-fbb6-4077-a6fd-07f48a1a70ce.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/551bc328-2cba-446e-8479-8191b2c9c111.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/682238a7-7bb2-43f6-be16-c325769162f7.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ba4da6d-41ba-437a-bf0b-2d763af79a8d.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d688b343-2fd6-4983-be54-492c83c9ce7c.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b6785f9-c6af-4eee-88f8-515a5a6df402.jar
May 26, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1ddad06-e643-405e-b72b-27753131eea1.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e9f6c0f-e4dc-4e71-95eb-edbb51879531.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc106217-a765-4518-81e7-1cee8f970007.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b63f7807-c7c3-4d82-b5a1-89162497430b.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85111cba-be25-4c6e-8acd-70a9f6da3415.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/26a000c4-7c52-4f89-b975-d2d73dc057a9.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/60c5b186-13b7-4c03-96d3-895887db3fca.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cbc58ee1-a4b4-4101-81fb-15e110e27d5c.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/17cb7294-2b2b-4e62-8cf9-4f9324959eac.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9218161b-c735-45b4-b208-049c59d08c8c.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/be2dc010-737a-4840-a0cd-c21645a7ba8e.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/55d4ea72-f503-420e-82b8-425aa1fdf09b.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ac568cba-6750-4c2f-952b-0d8b30b0d941.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e6112a2-6f1f-4a37-915d-4ec72ffd094d.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3768d926-b7e6-439e-9ca2-9d0fd0629354.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1261bf3-a224-4c1c-b696-bc4451c49b48.jar
May 26, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8e3100b-f479-498e-8ea8-2fa9441df5fa.jar
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 26, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 087f498e02894d033513077f729a5db44745c1976e5079fdf940da693a6840e5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7deb9ab8-2a66-495d-9061-30e181561426.pb
May 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 26, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-25_17_45_32-1915671427683872591?project=apache-beam-testing
May 26, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-25_17_45_32-1915671427683872591
May 26, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-25_17_45_32-1915671427683872591
May 26, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T00:45:32.678Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.021Z: Worker configuration: n1-standard-1 in us-central1-a.
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.629Z: Expanding CoGroupByKey operations into optimizable parts.
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.683Z: Expanding GroupByKey operations into optimizable parts.
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.713Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.804Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.839Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.873Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:40.904Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:41.364Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:45:41.449Z: Starting 5 workers in us-central1-a...
May 26, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:11.044Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 26, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:11.075Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 26, 2020 12:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-26T00:46:12.034Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 26, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:16.459Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 26, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:16.500Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 26, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:22.022Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 26, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:35.032Z: Workers have started successfully.
May 26, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:46:35.122Z: Workers have started successfully.
May 26, 2020 12:47:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:47:08.630Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 26, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:47:08.996Z: Cleaning up.
May 26, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:47:09.094Z: Stopping worker pool...
May 26, 2020 12:47:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:47:58.076Z: Autoscaling: Resized worker pool from 5 to 0.
May 26, 2020 12:47:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-26T00:47:58.128Z: Worker pool stopped.
May 26, 2020 12:48:06 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-25_17_45_32-1915671427683872591 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 673ff9e2-5b43-4d70-bf76-3f7bb9d6687a and timestamp: 2020-05-26T00:48:06.136000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.368
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 26, 2020 12:48:06 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 2 mins 52.061 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 46s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/mubh6d7tjefyg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/547/display/redirect>
Changes:
------------------------------------------
[...truncated 346.44 KB...]
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4dc650e0-bf5a-4ecf-a46e-b495fdca8775.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4f1957c0-6ab2-49a5-860e-f03cf2ef7c31.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/521cfa22-5a6f-4b5b-84d1-4d7b51c89d65.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/630aa4cf-51c9-4311-b30d-11db8e0b52b3.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33fee347-b09a-4d0e-9110-a038060d5449.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f7be4008-4968-4417-8be1-dc5fbfffe850.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38c31111-7597-4337-88b2-65c3ebfdfd89.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fbdd1210-ec24-4bf5-836c-d733988a0a66.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a298db5a-4a60-459f-b373-5c3a25001ede.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3614d19e-e473-40d2-8b8a-88feabea0c8c.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cfc13dfe-126f-415f-a7c4-714b7ad3bfec.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a85bd49-0157-4060-ba28-f27f5cfc1e0b.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/267f591e-f57e-43a8-9120-7a75d86e572d.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3b2d02b4-c4b5-49e6-bdfe-7fa50f0430d5.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/72089276-f383-471f-93be-14e9b5fe5f97.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3a29d209-f692-43cd-a305-acb4553d838e.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/09ebf099-b9b5-47bd-87a1-c9c21a029bae.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6695edf4-0574-4374-a0b9-62284878cc32.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/84642262-d9ee-4a5e-98bb-70d0b08a7610.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0001404a-25dd-4bb4-ade5-d35f470f0da6.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dffbafb9-e850-4d7b-b506-4536756ebddc.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c29af26e-0a6c-487f-87ec-9f32a570150b.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8714181-c637-4168-9d05-af660315dac4.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/81ba5120-2200-467a-ae4c-4a0e78c1f5df.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd5003de-304f-41af-bb79-ef0e6e9ab4f8.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7aad8947-9cf2-4966-98a1-39fe021f4ec6.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9f14af70-0075-4325-a2de-cff45a48d38b.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a4220d3-6303-4f02-8797-08f2509dd082.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b670a178-8df7-4c30-9ca8-52b1aa4d0f72.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9eb08d26-c0d2-4000-8135-d437d583fcb8.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a6daa043-ceef-4d5f-84c0-fd0d8c4c08e2.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a808faf-0fa2-47cb-a7ab-cf4759ef140f.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7435b6ae-fd10-4c45-894d-809c12043d54.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce8087a5-481a-4d99-9915-38c948bc1987.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7c843825-a13d-42b6-9c30-9a2de3f7e08d.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/241b3b77-d58c-4883-9bdf-1f3dbc8d0d7f.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d4f1d82-8ed2-47f0-9005-a13b91cadfe1.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58b6a8ea-21f4-4a1d-ae53-d8e5f47cbbfe.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14724db0-b982-48d8-abe0-8bcc5093de0a.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a19722a-c7b3-45fb-95bb-14f61c4b8944.jar
May 25, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/802b32d3-e2cc-4fdf-9a23-80d0803b2f6a.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/290b8ccd-64fb-4363-8ac1-f7f328c1cd50.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d15e7ebf-c015-4f16-8e62-8f6089c89b8c.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/46b9038c-4199-4067-8098-b1e5a3c90d7b.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b22a819-64eb-438f-9adb-a231965cab59.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c2d07081-aec3-4182-9d2e-aca0651889f5.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7f7c6144-7a6a-4c46-a121-d26316b5ac82.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ae1b643-fcd2-4f42-80ca-186a4c7178cf.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa98112b-ccd9-4783-afad-b01a42d65832.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0cf82d81-5e77-4fc5-a3ab-471d34f1d72e.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d98a172a-bd5c-4fed-8b8e-d7b3c9797d88.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2125919b-63ff-4023-8854-30af5f3c6d38.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61f6abff-113a-455f-8abc-370ad22ffe52.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/874411e1-5228-4046-8a06-4d7aac4f300f.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/98524e18-3546-4d58-aab9-3168e266b3e1.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7d13adc5-9c0e-4fa0-9897-65eafd074cd6.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae4fe5f4-29fd-4621-991e-cea0b7800c2f.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3d73fd3f-a762-4fad-9b55-910c137839a6.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/57247bf7-00d1-4494-8f30-6772841c1c13.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/acc0ac26-b3f2-4d76-bb26-8791a112eea2.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8fa14f62-ea01-4ef0-b286-003f329b2231.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9f1720b4-5407-4986-9996-70ddc54dba9c.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f378a02d-5f6b-436d-99e7-0300056ec893.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff54c38b-d2f1-46dc-8728-61e50cb02c40.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/01887a2f-6676-47ca-a385-4e895a2fad0c.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/63743c72-dc06-4a96-bed9-89e03ae3b5e7.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1148fbd-faca-4d9c-b1af-0422f5c94ced.jar
May 25, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b16d5770-4597-4176-a202-01097bd9584f.jar
May 25, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 25, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84254 bytes, hash f67e3df40ea576039e24426cfb5a7a093730a6d741ab41727547d152b90057b3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-a5004aa6-6124-4fde-885e-d558bcf5b80d.pb
May 25, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 25, 2020 6:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-25_11_45_26-2533355323362774406?project=apache-beam-testing
May 25, 2020 6:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-25_11_45_26-2533355323362774406
May 25, 2020 6:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-25_11_45_26-2533355323362774406
May 25, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T18:45:26.944Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 25, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:38.046Z: Worker configuration: n1-standard-1 in us-central1-a.
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:38.888Z: Expanding CoGroupByKey operations into optimizable parts.
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:38.981Z: Expanding GroupByKey operations into optimizable parts.
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.024Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.124Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.162Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.191Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 25, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.225Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 25, 2020 6:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.599Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 6:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:45:39.687Z: Starting 5 workers in us-central1-a...
May 25, 2020 6:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T18:45:57.497Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 25, 2020 6:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:03.756Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 25, 2020 6:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:03.791Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 25, 2020 6:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:09.160Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 25, 2020 6:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:24.124Z: Workers have started successfully.
May 25, 2020 6:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:24.160Z: Workers have started successfully.
May 25, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:54.766Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 6:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:55.057Z: Cleaning up.
May 25, 2020 6:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:46:55.132Z: Stopping worker pool...
May 25, 2020 6:47:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:47:36.379Z: Autoscaling: Resized worker pool from 5 to 0.
May 25, 2020 6:47:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T18:47:36.424Z: Worker pool stopped.
May 25, 2020 6:47:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-25_11_45_26-2533355323362774406 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c3816109-54d7-44fa-ae70-ff6418d1d288 and timestamp: 2020-05-25T18:47:42.554000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.899
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 25, 2020 6:47:42 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 2 mins 34.107 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 23s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/haiuv2dyefb56
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/546/display/redirect?page=changes>
Changes:
[kamil.wasilewski] [BEAM-10058] Provide less strict assertion to make the test more
[github] [BEAM-10050] Change labels checked in VideoIntelligenceIT (#11775)
[github] [BEAM-9930] Update blog post for new Beam Summit Digital dates (#11772)
------------------------------------------
[...truncated 347.27 KB...]
May 25, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45bf3861-f710-41f1-a9cf-96d3aedaf2cc.jar
May 25, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a85892d9-97ec-48e9-8ab8-ce8b1478bc8d.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc665ea2-733e-49ab-b67c-8b7e3cec19de.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/39674953-fa39-4fcd-8747-812c16371abc.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a4d6dd2-990d-4b86-914b-28955ce6ceac.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8dc81fd-395d-463e-9693-37120912c2d4.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4e40c143-3e3a-4192-91ed-dd0146f487d8.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/05fd97b6-6154-4399-a323-e964af89ac8a.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a4a9e31-c11f-4ce8-8001-57ee1db978fa.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dbd9497c-3271-4b2d-90f6-20311aaf0a1f.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2977703f-74cc-4b58-9be6-e3e58ca83062.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae992750-c909-4e60-9d0b-b868708b1c9a.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6378a2f-588d-401e-9057-3244240c27be.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ffe5ae74-0fa7-410c-8016-a8f3b70b70fc.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d345fa9d-a89a-49e3-a733-fd932e9f15ff.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1b78959-fd29-43bf-b9a3-cb112e2ccc96.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/125d1ad0-ca8c-48aa-a161-664fdc4c12d4.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a96be80-3428-4cc8-8616-5658ec4eb059.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bfdaf0a-7ab6-4fb6-a007-3ea51a33ee80.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04f85c76-5441-4fdf-8d90-43878ff57872.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e5f5b8e-4d05-418f-810d-b2a0d1ded287.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9476fc8d-db92-488c-b2c4-db53a6097aab.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9879aa3a-f9aa-43e5-b87f-51b62508d2b7.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5134223a-2980-4a21-98f5-5ac7c7d287d9.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/262a9610-a810-4dbe-baaf-3def618b60db.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ec9f719-a6a8-4aa7-8721-0db102563ba2.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d290238f-5e59-44a9-9669-c66d60d69abc.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bcf292ce-48d8-4f17-8e16-ed15557da009.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/81d68678-35b2-4370-86fa-bdb400103ff1.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5162caa1-39d0-449c-a291-3b23f1ad40e1.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2c0a9d36-79d6-4a5f-8181-b2cd8035e340.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b9910111-0515-475a-9b54-546bbc478995.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3102cba0-aa37-420e-99f8-531b4d1aa4fa.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd71d1ef-0b04-49f2-94a9-438a13f421f6.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d512faa5-7749-41f3-98a8-417c08570846.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6085bdc-474e-4411-99aa-ccabf94eef3b.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d98d8433-ea9c-44ea-8ec5-b5120a78a08e.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/233869c3-f42f-42e2-806d-5ceccd5a0178.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4ed9bfb5-60ad-44b8-96f4-4ff6cdbcbd97.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d19a4856-30ae-4f75-bb82-cc744df14692.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49e40445-524b-4864-b862-b2fe98600136.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e287180a-7e6d-400f-9f5f-306ac9a1d77c.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/60f573bf-3295-44ba-93b0-944225e59c4f.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e40d7b1-e26f-4dd2-84e2-7fd085207fa1.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/177f6326-d0d5-42e0-8398-d695db411ca2.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cda900f4-48fe-4d4c-83e8-15a1d1c83285.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/368f0929-551a-4d1f-b710-4e1707491352.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc2e6fe8-95d6-444d-919e-f1017ed49fb3.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/035d2603-b8cf-4b9a-8b92-943edffdfb72.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b407e07-388d-48c9-b28b-b38e8421e1e6.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/79a955be-4470-4099-9168-cd28098319bf.jar
May 25, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eeee1b50-84c8-46ee-a2b0-742773b9e455.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d01454f8-ec48-47f8-81be-4411b2fbec1c.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ec1150d3-1eb5-4d70-a1de-f16703274243.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/543c8cac-ff5a-4234-bd3d-66639c7f3ea0.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ddb980c-8a7d-47bb-91a1-a814642abae3.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bfbdc7b-e148-4c4c-a5f4-975a3677a774.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b3818030-6066-4f20-a7da-2730f58ac961.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/197bc463-5d31-448f-ba9f-68b351c41705.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e542a911-3207-43e3-b2ec-d56236849b01.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c79ac013-decf-48a8-b49d-d54ac7bc95ca.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f84ab25b-cf2a-4fe3-ab7b-3074ec8f8fa0.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de4fd6c2-2bbc-4326-b042-1bf77619dca4.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa82ff20-04ef-473e-a3f0-cf42b7bfcf9b.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b01d37f-b967-40a6-91d9-e27333bf5d4e.jar
May 25, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cdb18a78-5074-4238-92fe-bb2476543597.jar
May 25, 2020 12:46:44 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
May 25, 2020 12:46:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 82 seconds
May 25, 2020 12:46:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 25, 2020 12:46:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 25, 2020 12:46:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 25, 2020 12:46:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 25, 2020 12:46:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 25, 2020 12:46:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash bf0c1d36a17e8fde61a671b83396ad57c3cb9bab3aae2ef91fee023e9786b109> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8aeb779b-0934-40df-b7c1-e214796832e6.pb
May 25, 2020 12:46:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 25, 2020 12:46:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-25_05_46_46-13571415204873288485?project=apache-beam-testing
May 25, 2020 12:46:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-25_05_46_46-13571415204873288485
May 25, 2020 12:46:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-25_05_46_46-13571415204873288485
May 25, 2020 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T12:46:46.897Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 25, 2020 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:50.242Z: Checking permissions granted to controller Service Account.
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:54.461Z: Worker configuration: n1-standard-1 in us-central1-a.
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.142Z: Expanding CoGroupByKey operations into optimizable parts.
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.175Z: Expanding GroupByKey operations into optimizable parts.
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.197Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.278Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.303Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.331Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 25, 2020 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.361Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 25, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.743Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:46:55.808Z: Starting 5 workers in us-central1-a...
May 25, 2020 12:47:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T12:47:05.568Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 25, 2020 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:47:18.622Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 25, 2020 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:47:18.654Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 25, 2020 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:47:24.027Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 25, 2020 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:47:39.103Z: Workers have started successfully.
May 25, 2020 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:47:39.137Z: Workers have started successfully.
May 25, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:48:12.466Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:48:12.703Z: Cleaning up.
May 25, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:48:12.806Z: Stopping worker pool...
May 25, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:48:57.495Z: Autoscaling: Resized worker pool from 5 to 0.
May 25, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T12:48:57.541Z: Worker pool stopped.
May 25, 2020 12:49:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-25_05_46_46-13571415204873288485 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): df351196-3c19-4592-b825-221bc534132f and timestamp: 2020-05-25T12:49:04.621000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.45
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 25, 2020 12:49:05 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 53.042 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 44s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/ecn65ieggwioc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #545
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/545/display/redirect>
Changes:
------------------------------------------
[...truncated 346.16 KB...]
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c5e42e2-54b8-4254-a4a0-1d45cfb6a6c6.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08adaa9e-9270-4926-ade4-383ccbb758ea.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cee84f44-dba2-4f0e-b747-4fe433196aca.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8991c04c-d174-4ae2-8bca-755e5bc5f918.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/75486c86-7c60-4b7e-ae81-f7e353c860ab.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35403663-94cf-43b8-81ab-5898f24431aa.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85d65748-9188-4f6a-97a2-43626426571b.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5bcffe0c-ed3b-4174-a2f2-84bdc1cfff65.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a359de5-908e-488c-9bf6-3fabdfce173b.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/37304a70-8754-44d7-89b1-dfbe72b43139.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bc433ae-0095-468d-82fc-803e53d7ac62.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fad48f97-f296-4b5f-8056-fed2517e279a.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff91c283-3944-4053-a48e-85f3a2f8bbd8.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7ca54676-f080-4bab-b5c4-f0f866e67c81.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/693fa3cc-1378-487e-bc0c-9f0dd9ea0e1d.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2501aed6-bbb4-4f4c-95d0-8e967946daf0.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eb11d9bc-d319-4bdb-838d-b1611ce8eeb2.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e62a696b-16ac-4491-a8ce-c75c53e30a46.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/893a1afb-b921-487d-b9f4-f9fe41a8682b.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d972cc1-b020-4824-9fd6-885bc03b1af5.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/853b52d1-6c68-4b21-8b9a-ec5d63eeb252.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/742ec68e-aa7a-4a93-adff-9bbe32240b79.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e4785b55-1dc0-4242-b271-dbc089897ba1.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/deb6a461-eb68-4e26-a359-9c26b4a6c149.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c0099e1c-61f8-4997-8e34-b6f8b544dcc8.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b80f1c38-74a6-489b-83af-3de29095ed03.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c88d2e1-f93a-45eb-a62f-632b929bbea5.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16f02cf6-aa49-4a58-ae2a-f22346fafd3a.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5dd449d4-c64a-4967-8925-e6eac89746a2.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d7840df3-b9f8-47fd-81a4-f17120d9f4a1.jar
May 25, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2c53a61b-eec7-43bd-826a-b1c601ebdf61.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c2cecf87-6bc7-42e2-9816-8c61776f9ec4.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0aa42d2a-3f72-4f19-8021-b93a84a55279.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/653cc47b-d69e-4ff2-92ed-83b8c60b475e.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9fafde1d-0fbb-4d34-9f74-51b965bd8014.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6a67f3c0-7278-4580-ab6d-141377c3235d.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c0a1fbda-9e20-4252-948b-db4bc999652d.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f748def9-d4c5-4a75-a1b4-981f84c7b20a.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/105c78fc-718a-464a-8189-e08836ff08a3.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/15d4635b-4df9-4217-9524-f633aab79164.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e5f4320-e840-4a70-8fb4-d45867dbb188.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e713e52-a30a-448c-9911-939d93080427.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f196ae93-9a94-4bee-9715-43cc30beae0e.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c8052061-0a5c-4a7b-a3dd-6300416908c3.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/090e1586-7e1a-4d04-8bc4-82c800e4fd34.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a716fc9-0e43-48ec-bfb9-0308fda00448.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1b2e53b-95e6-430f-afe2-15617e4a66a2.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fec303c2-ceb3-4c95-b74d-fed1839a8de1.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0858eb9f-dd60-49b1-b1b9-67ea5b8724ac.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edab797a-ab01-4031-ba8e-b3d0fb8ec1c4.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dbc682c9-5f7a-4174-b4c7-c3d900d2df81.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9464967-f118-419a-88db-126b72911cbd.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4be22d0c-02d4-4f3f-8dcd-c7a6e98cdd98.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0c9d0f39-c23a-455f-9d17-7ed328157312.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/785e8c10-d3a2-4d78-965b-165e72f8fee3.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2597ee33-dcf6-4aa9-bd43-8c10ee5174c6.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3bdec871-5cb4-48d9-9dd3-404f64b3242b.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/22629bb4-09a5-488e-a04d-02534dbc1976.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2bfef232-f4d2-4c28-addc-98ae37264f92.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3d75bef-1701-4f51-a1d8-d25c681febd8.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5eb7f91f-dc69-4813-88f1-f2802a4ef44c.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/76152b54-ccbe-45d8-b787-13768adb0424.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aaaca9af-fe47-4224-b343-f689d5d37598.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2ed59f56-d303-464e-b915-7411599f92dd.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba7a7971-275c-4a97-8ee7-8d4bfa67e8f0.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d4fbaf46-6c2b-4c31-8285-086454e6bec0.jar
May 25, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e276202f-30f8-4a46-af36-f2fa0122b5fc.jar
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 0516ce2623f5a404681e0652538677ca9de87a9d38ae74b7692ee9e0d3d8c74e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bdccd347-cbe6-4d90-8ae3-16f95c5c8499.pb
May 25, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 25, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-24_23_45_26-2355320189167856331?project=apache-beam-testing
May 25, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-24_23_45_26-2355320189167856331
May 25, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-24_23_45_26-2355320189167856331
May 25, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T06:45:26.043Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:30.018Z: Checking permissions granted to controller Service Account.
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:34.449Z: Worker configuration: n1-standard-1 in us-central1-a.
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.085Z: Expanding CoGroupByKey operations into optimizable parts.
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.127Z: Expanding GroupByKey operations into optimizable parts.
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.156Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.237Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.278Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 25, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.313Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 25, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.346Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 25, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.672Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:45:35.754Z: Starting 5 workers in us-central1-a...
May 25, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T06:45:43.024Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 25, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:00.065Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 25, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:00.100Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 25, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:05.471Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 25, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:24.175Z: Workers have started successfully.
May 25, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:24.199Z: Workers have started successfully.
May 25, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:55.869Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:56.091Z: Cleaning up.
May 25, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:46:56.171Z: Stopping worker pool...
May 25, 2020 6:47:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:47:41.993Z: Autoscaling: Resized worker pool from 5 to 0.
May 25, 2020 6:47:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T06:47:42.042Z: Worker pool stopped.
May 25, 2020 6:47:51 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-24_23_45_26-2355320189167856331 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0d4403c8-48a5-4937-95dc-9dcafec22b5c and timestamp: 2020-05-25T06:47:51.367000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.527
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 25, 2020 6:47:51 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 2 mins 43.854 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 34s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/sl6spshepipka
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/544/display/redirect>
Changes:
------------------------------------------
[...truncated 345.83 KB...]
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54bcf0a3-ffbc-447e-9a11-1cab49491b4e.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0551b3f-debe-4520-baae-76b7ae03f626.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b28f9f17-65e3-44bb-a85c-a6b1ded5406a.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ca47f79e-fa84-4d8e-9977-4930f24f5cb7.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/795b3cfa-a7d1-4f54-9ca7-8af5812cbe80.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fad12006-b720-4f8e-af92-c923ec7b8afe.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2ececc32-308c-45c2-af29-022164330522.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e7e016cf-9e30-4d72-89d6-9858c52634a0.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f389b627-d03c-4e01-b3ce-cb9435fbd79c.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e359a755-b07f-4a6a-821a-28ca1f20a6f3.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a7ab9d0-53e8-4167-889d-c959318faf99.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07659923-cb91-478d-a0be-3478153354be.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6631228-f8c2-46b9-8ef7-6e36219bc83d.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/46de621c-0887-4455-93fa-eb8c97a76b38.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c4be66a-62f0-45e8-b8a9-2823b056c4e9.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31f5be93-81f9-45de-8222-c0d51ddafb2f.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c8263a4-f054-4a72-870a-9aa11a6e3834.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/46e2533e-fe86-44b8-9078-729a4a5efd57.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6b5ee68-7131-4bf3-9a74-b8ab7d9ac962.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/81d8470d-e577-42d9-b919-7481c8f2f659.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d52b11e-9575-46ec-8430-dab08831e4f1.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6651ebf4-3bc0-4d41-8296-5b455ec24db8.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/acdba49a-579c-4bda-aa97-0430b9464337.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd75fbdd-fda9-41eb-8cd3-7a2666c2d763.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e01c14ff-2173-419c-b9e1-0afce077428f.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87abe114-46c9-4dae-9c08-4776b6407041.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35ed999d-693f-4911-b901-3f6ab7f3ac4a.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a199b19f-bed4-493a-b052-8db869e2a163.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5d23f1f9-2d7f-4511-92e0-39f1845b62f6.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ee6854a-9cb3-4165-8f4d-9daba14e933a.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d55e6da-daad-4c51-8188-537bbd276702.jar
May 25, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1a40806-e1a2-4165-8e26-b845fedab85f.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/15913bc1-dfda-4a28-9808-cda95996511d.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f785ec0-63bd-402d-8cc6-336c96dfcd1a.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c19a571-f4d4-4451-997e-08b5065a9eb1.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/78eff843-3fd7-49bf-9a79-c5b07f1508ff.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/338a4ea9-d410-4367-ae96-94c9b0c0c34a.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/da49324c-5768-4fc0-99c3-30c81cd79216.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/84409c5c-85fc-423f-ad1e-4ee408fbb463.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b469e52-6aa3-4294-9f28-f0a0f81487ee.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/93e68b95-cf8d-4dae-b8c5-e4887e0db06b.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2f8c4a53-c4e6-4145-a369-e31a9fffc474.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eba1ca27-50a6-4150-bae9-d36f45d6f54b.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e8cbc1ce-e75b-4829-9582-0726ed9d150a.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/129593b5-f60d-4539-b895-036ae126acc9.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3f00204-53dc-42e8-8f75-e8f6bb732a92.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/abe311cc-8f74-4148-b6bc-fc69dabe8daf.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/523d93a8-838f-4265-9013-f5a7068c837a.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/48c24078-5ff2-4732-b496-bd896f684667.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a84609e-e237-4ce8-815b-d0cab257c778.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b8b493ae-222e-4be8-b068-b7de621a4299.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8da6a64-bf3f-4f4f-9a6d-cfe6f63eda65.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ad00c6cf-a7e5-4d95-be89-0235546ee01e.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3a529638-d2d1-4b79-937b-1cc072cf4562.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ea6bb5b8-b505-4701-ab67-2a269317b242.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b8cddcdd-4fe6-4d21-b5d5-a113bd1ff26a.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0488a325-344d-4745-9b2b-b40e4d1b9e3b.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b4e3915f-da5d-459f-86a5-0c0a6713c802.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3da76e49-edec-4935-8772-c411f99d1c84.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d76adac8-78c2-48a5-b788-7631f1a26b79.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4d1dd7b4-729f-42fc-ba48-2717a592f45c.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/55938972-ed11-4129-b121-7b7d17f37f5d.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d67d3540-8cf9-4830-9304-3995306d0600.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8846a1ae-62fe-49e0-b5b7-ca926a4bfa0d.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/946ea78a-f7a8-45e3-bbec-b46ea52e91a4.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/505639fc-6cc8-4aca-864d-4eedf40c0177.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/934b98b4-a4e3-4045-9e58-efd6ef079353.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a87087e-aca5-4bd5-8bfa-e0d97d9318e4.jar
May 25, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/36a201dd-2e99-4b94-a440-77cbe17729ae.jar
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 8f8b22b0123ba7bbd9a4e05a24e04d2fea6acb128e3b8c00c4d0600d63767c7d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-b4128045-87dc-4d59-ba19-58a3db90f8c5.pb
May 25, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 25, 2020 12:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-24_17_45_35-12630377997455454081?project=apache-beam-testing
May 25, 2020 12:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-24_17_45_35-12630377997455454081
May 25, 2020 12:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-24_17_45_35-12630377997455454081
May 25, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T00:45:35.932Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 25, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:39.067Z: Checking permissions granted to controller Service Account.
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:43.543Z: Worker configuration: n1-standard-1 in us-central1-a.
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.099Z: Expanding CoGroupByKey operations into optimizable parts.
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.128Z: Expanding GroupByKey operations into optimizable parts.
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.156Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.218Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.236Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.261Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.284Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.616Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:45:44.680Z: Starting 5 workers in us-central1-a...
May 25, 2020 12:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:46:09.748Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 25, 2020 12:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-25T00:46:11.622Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 25, 2020 12:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:46:24.516Z: Workers have started successfully.
May 25, 2020 12:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:46:24.543Z: Workers have started successfully.
May 25, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:47:04.793Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 25, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:47:04.979Z: Cleaning up.
May 25, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:47:05.074Z: Stopping worker pool...
May 25, 2020 12:47:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:47:49.719Z: Autoscaling: Resized worker pool from 5 to 0.
May 25, 2020 12:47:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-25T00:47:49.757Z: Worker pool stopped.
May 25, 2020 12:47:56 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-24_17_45_35-12630377997455454081 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 669c8eae-94d8-4a92-bdb1-c2ea661d1a56 and timestamp: 2020-05-25T00:47:56.170000000Z:
Metric: Value:
fields_read 4375276.0
read_time 23.156
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 25, 2020 12:47:56 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 2 mins 37.655 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 29s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/ln5vkg5olxkvi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/543/display/redirect>
Changes:
------------------------------------------
[...truncated 345.92 KB...]
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d113dc5b-1f48-43e2-9c9c-2b8833bbfcfb.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58a3c13a-7367-4a62-93ef-604f0d8429d0.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14fe204f-4858-4be7-9d08-131135c1b538.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd223b16-92d6-4fa4-bd05-347e8247761e.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c68dee7-c015-45d7-afac-a8bb07feb5eb.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14f67a60-2c60-4d2f-b0ed-067dfd4a058c.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5f377db7-f3ca-4f2a-93f0-1a6e1b29fd26.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/93e306a1-56e7-4eb9-b09d-835930dcf3d4.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2623faf8-ae33-49a5-b316-43016c9e38db.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1c3834b3-41e7-4764-b910-bee2aa6daf1f.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d58d094d-a566-423b-8d29-2fdf589bae1d.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c44b17da-8552-42a4-926f-20d9004f59c8.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7136f023-daaf-44eb-bce7-2945377b799d.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a574f4d-90d9-400a-8771-7093c9c35c05.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe37a3f3-7bcb-47ad-bc8a-5341d14fb3fe.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/78e37274-99ce-48f5-a1c0-3e8188188348.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/43d0af14-3228-4a0e-a7ed-be628ebb00ed.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/12432653-25c1-41a0-b8e3-356383be57da.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d5444eb6-f06b-404e-8a10-657b0e8d0df9.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df97ef0e-97fc-46f1-a708-37d05652e178.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/794dde10-d0b3-44ea-b930-77b595d8c9c5.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ddf05aeb-edd8-423f-be61-944c8b332014.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7bc4ebe6-6270-4485-a8f6-b0a98b7e46cb.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b7bdbd3-e93c-45c9-ac71-9b51dc1794cd.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8df6c2f6-eba1-4755-85fb-8b62b33fe56a.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa5cd428-72cf-4688-9cae-679c5664ee4b.jar
May 24, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/554bb853-e1d3-4907-830a-76d25e22d010.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c0d478ce-f49e-4902-9b5b-824bf4d19c8f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b795774d-b96b-46d1-b137-2669b5c94d4a.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d80dd2ea-70d3-49dd-abf5-ac574fcb5bce.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/65003395-4d33-4b96-bb2d-f8fb525280d7.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0776b035-8d6c-4e09-a53c-24d7ff77be81.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a788f692-b014-417f-8d1a-a56c87a14b6f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bd24526-c501-499f-a6d3-cecd44a39b0f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d723679f-cb37-4a89-a6ae-0f4e1622ff08.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/488cff93-6afd-4e58-a7d7-b2744f9437c2.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4998e7be-e2c3-4f3f-9cc8-f4204175526c.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a021a35-2c9d-499b-8824-92103beb8b90.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6d17875-db33-4a97-8c0b-acb7d7c368bc.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/53c7fa37-f4fc-4804-a70d-035769d3c2c7.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd0d3dd7-5e75-4506-8bea-140e7a0f01b0.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7cb708fc-2bb3-4a7b-964b-23ad580ca3dc.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8683495-156d-461f-960b-73c7b18f1d4f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6711be7-3f82-4043-9cf5-7bc71e8f10c9.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/abb610de-2498-4a7e-8dd6-153686db9532.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/13d71107-d119-4f61-8563-dcd7e658cd64.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f74e3d2-b85f-4687-8a07-ea426424557b.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a054b57-accb-408a-8387-2fcbdb5428c0.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/620cdd66-7eba-4cf1-9d4a-1fd942598f80.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c759542-12ae-477d-a9b7-df122856d83e.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d5d0c8ee-0444-4c1a-a9fb-8b74f7721651.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/afa5d6af-f25b-4d01-bf07-62a75592c73a.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/50d05528-565c-4270-a045-5f736df08f68.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/20800772-0561-4b2d-b311-ec530ab52d2f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc846af2-1b4b-4c36-9f5b-055b8fc601f9.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd636935-9b77-46fb-959b-e848a8eaa742.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3a9261f2-d5a6-420f-bbf0-a43337fe730f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3d09f15a-dfdd-4bf8-a02b-55a3b241767a.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58692aaa-1335-4829-9827-10464e97977f.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/59cfb3f6-c627-42bb-b8e5-e3284299ac5d.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/02a8b4a0-b2a2-4e69-b9f6-6cdce00bde4a.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ecdc43e-d17f-48d2-9e50-502b181008c3.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/56439408-20f7-46e1-a260-c1a2b81635f0.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f33dcb01-50e9-4138-9537-a8db976cf5e0.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a6893aa-6251-40a8-b3e4-378659b29956.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/012042e8-ff53-4aa8-a196-affba9b203fa.jar
May 24, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3d3dfd82-f98d-41d6-8233-815a28e6ea76.jar
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 8c525c53255318de849b9d077f9a6c535dd0c6da56e725be6ad3aed073874072> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-d8c723da-6fe4-4480-b5c0-4e8c0a8ac4a3.pb
May 24, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 24, 2020 6:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-24_11_45_28-5268296159930212137?project=apache-beam-testing
May 24, 2020 6:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-24_11_45_28-5268296159930212137
May 24, 2020 6:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-24_11_45_28-5268296159930212137
May 24, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T18:45:28.051Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 24, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:30.894Z: Checking permissions granted to controller Service Account.
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:34.408Z: Worker configuration: n1-standard-1 in us-central1-a.
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.117Z: Expanding CoGroupByKey operations into optimizable parts.
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.148Z: Expanding GroupByKey operations into optimizable parts.
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.176Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.237Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 24, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.258Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 24, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.284Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 24, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.302Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 24, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.586Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:35.647Z: Starting 5 workers in us-central1-a...
May 24, 2020 6:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T18:45:56.236Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 24, 2020 6:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:57.085Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 24, 2020 6:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:45:57.111Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 24, 2020 6:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:02.509Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 24, 2020 6:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:15.216Z: Workers have started successfully.
May 24, 2020 6:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:15.238Z: Workers have started successfully.
May 24, 2020 6:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:49.151Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 6:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:49.301Z: Cleaning up.
May 24, 2020 6:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:46:49.358Z: Stopping worker pool...
May 24, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:47:24.758Z: Autoscaling: Resized worker pool from 5 to 0.
May 24, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T18:47:24.793Z: Worker pool stopped.
May 24, 2020 6:47:29 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-24_11_45_28-5268296159930212137 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f7600662-59c3-45bc-aaea-e91057ac462d and timestamp: 2020-05-24T18:47:29.783000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.385
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 24, 2020 6:47:30 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 2 mins 19.738 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 11s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/uyqrl37yqmaw6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/542/display/redirect>
Changes:
------------------------------------------
[...truncated 346.88 KB...]
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ea8d5436-9730-4f3f-8249-0d830942fa9d.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/97069b3e-5196-4b73-a3e4-225aca2dadc6.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae59f0f4-74f4-4bee-9bc1-23fca926a23d.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/735af4f0-3332-4e6c-9f9e-e1c330e11b4c.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/67fb8da9-e8af-4c02-8ff2-47ed46368299.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14631217-9e32-4a1f-8fe8-482234d8f33e.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1b3abd7f-a0db-4373-8dc9-9283d01bb4e5.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/313f99b3-7fc5-49cd-a1e3-7b4599322ed9.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e0674c3-4a74-4488-911f-ec0fb7d64748.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d71bf249-0014-4d24-8af6-6e4d7c3159e7.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a818ae2e-be36-4aa0-8c7d-f5e44f88449e.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3af7ad57-e2dd-4d1a-b790-111051953bfe.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6b9b38a-3c73-4a5a-b3ec-8cb693f6ea49.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/37cb0fd4-70da-468f-9eb9-23d7272d0846.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3d6d9541-514b-4a1b-8f94-94e65be65e31.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.api.grpc/proto-google-cloud-bigquerystorage-v1beta2/0.90.0/c59fa6741ab5d84a2de4c11f07f41510751e52c4/proto-google-cloud-bigquerystorage-v1beta2-0.90.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e6c1218-9afc-4e27-86d5-efc1c00e4b5f.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e66c8dae-262c-4f02-863f-fa38b553c7ab.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b665d477-c0ce-4aab-9eed-1ef5269439a1.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/180eb5fe-f16c-4cfb-bfc3-c53c7503a090.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82a8127f-364b-4104-a1e7-1e86492caf6d.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc78f376-0d65-4b22-9483-07162dab19bd.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ca32283-fa2d-4e44-bf0b-6b7794c56044.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/255d5037-4de5-4fed-a788-5bee37dd7764.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fb298603-c40c-4584-9593-7052609049a5.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed950e14-7c8d-4ea9-b234-b6e59684b34e.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ac2cec3-ec92-42d6-bf4e-22d21e58f1c4.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3eeca55c-7ad8-4fc7-9247-a16cc9bf5015.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aacc372a-128b-4d2c-8894-ba94b5e366fb.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6af60f9-9e6f-42fc-b203-42b06785b3fa.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f856c01d-eab8-4a31-93eb-01292b03d85d.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae3a1f81-f7e6-48da-8b26-b8aafd23fb27.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a82348a4-ed77-4da7-a4ab-04d2de9c116d.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/806336b9-1a8c-4f5c-8e59-8d138d16c0bc.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/748bce1a-a856-4142-9750-fc90be72d341.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0728dbfb-7db3-4842-bd31-12542a191e31.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/40c30a16-d39a-4006-a96a-022744076ffd.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bf485202-0cb2-4474-b351-facb3f15d489.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a29adebd-18ba-40b0-9b16-7863e28e1eb2.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3a6943f6-eee7-4f05-a375-d74eacc2c2dd.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa5d12c7-df7b-4519-8a15-f069532474b2.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e6a5fb7-fe9c-4073-9d6c-bece77299457.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8bf697a-125a-499f-bf06-6de7b98ed69e.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6fe6f526-791b-4d51-97ed-03188bfa20e6.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0e74d292-bbb2-4e82-8bfa-c733af857f86.jar
May 24, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ac668d80-544f-4836-8919-aebe170f342b.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f7026992-f24e-4341-9958-af783e819ded.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c537896a-4154-429a-b811-85d5bd9e6153.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0a6815f-52c0-4bf4-8702-c56ff1ecca51.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3f5d839-5e74-4151-bdcb-77fc5f06134c.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88a87ca8-3e18-41d9-a711-46bfd0697b35.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45db3665-9e74-4997-ae10-5742fb508489.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82c9bf3f-1982-4d23-9730-814de6813bf6.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e376c13d-a32a-4fe5-858a-d3ab54b80e36.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2f02883-6224-4571-8e87-7ad1c72d9561.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ccc597a-6f96-4700-81a6-cc51fb463ef8.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4e8dc500-f60c-4235-b955-47372cfe62db.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/51bd189d-49d8-4c7b-99b6-d4d8a38475f8.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/47b001e6-752f-4336-bbe5-2929f9088a1d.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de2f9671-51fb-4703-a529-6aea162b01d3.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fcf44120-01a6-4b54-83ae-88fbcbeabcc6.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae4b807a-1cf5-4f4f-89da-75b23104ba65.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5f306021-e8d9-4c0e-9527-448b2977f4ac.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e26b5827-4050-4d39-9483-c2567270028a.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/32c9e4f6-e4f1-4603-adc4-3d4202d23471.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/617f7779-7eea-4974-97ce-39335e5617bd.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef618330-d82a-4b7b-9f60-be812d377e39.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/680b9279-d32d-403a-9b7b-3e8aab9e62ce.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bcea9961-7486-4f7c-9468-f9c9647f4e1e.jar
May 24, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1e714635-5081-45cb-8870-72d2a53c442e.jar
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 24, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 24, 2020 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 230f05766b9e63b9e2275ec2ecefd99396bf097b5a117da4d7cc2ba89bfefc77> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8739832e-845f-42fa-963d-24482022e672.pb
May 24, 2020 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 24, 2020 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-24_05_45_27-15152498730298838775?project=apache-beam-testing
May 24, 2020 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-24_05_45_27-15152498730298838775
May 24, 2020 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-24_05_45_27-15152498730298838775
May 24, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T12:45:27.615Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 24, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:30.681Z: Checking permissions granted to controller Service Account.
May 24, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:34.536Z: Worker configuration: n1-standard-1 in us-central1-a.
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.190Z: Expanding CoGroupByKey operations into optimizable parts.
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.230Z: Expanding GroupByKey operations into optimizable parts.
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.261Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.357Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.387Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.426Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.462Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.899Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:35.976Z: Starting 5 workers in us-central1-a...
May 24, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T12:45:57.517Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 24, 2020 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:45:59.411Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 24, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:46:17.589Z: Workers have started successfully.
May 24, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:46:17.626Z: Workers have started successfully.
May 24, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:46:51.949Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:46:52.179Z: Cleaning up.
May 24, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:46:52.278Z: Stopping worker pool...
May 24, 2020 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:47:40.528Z: Autoscaling: Resized worker pool from 5 to 0.
May 24, 2020 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T12:47:40.565Z: Worker pool stopped.
May 24, 2020 12:47:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-24_05_45_27-15152498730298838775 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e9d612f6-d136-40d9-9dc0-a2ee04f1cba0 and timestamp: 2020-05-24T12:47:47.103000000Z:
Metric: Value:
read_time 18.156
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 24, 2020 12:47:47 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 2 mins 36.961 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 30s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/h4jropdetyxc4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #541
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/541/display/redirect>
Changes:
------------------------------------------
[...truncated 347.41 KB...]
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f3ce775c-69a4-4b25-ab8f-737ff574e122.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3fd26a88-2320-4c54-a685-f30dd5eafd0e.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d27e2fdc-9293-459b-98e7-e3b0c3b06873.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/05301204-94d7-460d-9f81-04528a45f71a.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5d869696-f709-4c66-8bb8-d6f8e5a15de7.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3077a593-0938-4bdc-aa50-186c891e7336.jar
May 24, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ab9800e7-206f-4c6d-9b2a-62949e5c2f37.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e9db299-838b-4604-839b-23286d7684cd.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/95016a53-222e-4df1-bef9-f43f89ffc25b.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82c745f6-29b2-4295-904d-1a1e96d98100.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35084651-16b3-4f90-a883-6d551b8d858e.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae2d6d32-bd7b-40ca-936a-4833130bdabf.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c05c6ed7-9e3e-4c08-a2d4-74d230c07765.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/12474e34-fd1b-4551-9a45-f7fd5c869c35.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/69687d34-2c78-4f26-a748-b3d70017a182.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44526cda-174d-4023-b847-94b2c48d3f17.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b2ceb428-e7b7-422c-819b-305f835bef40.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/98d7a07e-514c-4c19-90b8-dff2274b8d08.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2505df61-f194-4d4b-8282-d7ce666a9703.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5f57fb73-b327-47f4-9704-e4bb7b1f5987.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c82a90b0-a4e0-4dc2-b9bf-be9a782441ac.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa5ff03e-9f0b-4e49-9245-2e7ce4c279ac.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ad6b0cce-7b81-4c7d-9888-1e137eb57aba.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ccf8bdc7-fc81-489b-a9be-3b7ddedfd17c.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/daefb791-3786-41fc-abf5-5bd55cad5740.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8bfd7f85-b0b3-4c73-8f6d-e8477a20012a.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/56abc0d7-a240-40a9-bf01-e8cb68fd1183.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6f155734-5014-4ec9-a477-2e5b709684f0.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7662abaa-0c12-4935-82bf-90da0a130a6a.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23071137-a918-4d9c-bd00-8480f5ecd2a1.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bf1792e-db9a-4e45-acf0-98895925d0ff.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e8adf05-9901-43ae-8d83-d4152c80cc32.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5733892c-8453-4696-857c-a109eb98b9ff.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b14dc30-0fe2-4a24-9a22-34276ef0e352.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06569587-c488-41bd-92ff-eadf7ab8ecc9.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f4477e0a-4b3f-447c-98ea-d0fa166161f6.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2be0e441-f152-4ed6-8556-1599ba004b9c.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23171067-e157-462e-904b-3e174622383c.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d421dd69-4943-4136-82a1-04dc24edf1fb.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c128468a-47f4-4198-b4a9-ff877a268158.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fb106f1f-f869-4498-8051-cc45c90a6985.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c4db83f4-4164-4007-8496-41e0e4c5d02c.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e4835272-c192-4e8e-a191-f3cf5adf8f68.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/194a87a8-a7b0-47d7-9cd7-0cda416ad220.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/204a6f5d-a5de-4e45-a5d7-b43dea00a37c.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/284bc171-0f5b-4127-9c90-a6f7cf6c59bb.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a843eb7-b837-4096-8018-c0c80b6bb11a.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f4e75eab-b10d-417c-802c-b6c0c3a90d4a.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba5fb075-2402-48f0-8704-0a8a5fb594e4.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c10f8b95-b7df-4744-b0b6-95197041671e.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a4df0bc-94e7-4d10-897f-1b61e3233e67.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a38df1e6-d546-4f19-9d0a-e71ffe234f61.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3e1f0a58-8a66-4708-8454-c44857fd08dd.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2f821844-9b61-41b4-907f-c44b4d60d8f8.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc13e37b-b806-40eb-9917-fdc46d9cde83.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/560104fe-be29-499a-819b-6421b7302876.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bbd8ea5d-5348-4c97-863f-2314c787d794.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4122e8b7-96b3-4787-9754-25bc4f0ab700.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6eaba81-549e-476a-addb-b0d0f2d1951b.jar
May 24, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4df59ff5-578d-43a9-9dc1-19deed443c5e.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38f0b0dd-38c6-455b-b36a-6ec8dcf24fdc.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2f852223-090e-47bd-9284-8a148cd09646.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25039998-dcfc-4d45-8d4b-d4eeeba79102.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f4adb62a-a1f5-4ea2-aa03-26fb7f816e2d.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/af033a1e-1969-4f79-b68e-ea19f89eb30d.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/20f13b6a-eebc-4724-b88e-086f0327351a.jar
May 24, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1001a341-358e-4336-bdd9-906fe4f02b1c.jar
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 24, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 24, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84254 bytes, hash 3053672f065546fbeb1b5074639ebbb340b4585904698ed72737a1542997e106> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7350f78b-3448-41d3-a7f0-8c192d2c3299.pb
May 24, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 24, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-23_23_45_26-1649232473479101083?project=apache-beam-testing
May 24, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-23_23_45_26-1649232473479101083
May 24, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-23_23_45_26-1649232473479101083
May 24, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T06:45:26.940Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 24, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:30.217Z: Checking permissions granted to controller Service Account.
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:33.622Z: Worker configuration: n1-standard-1 in us-central1-a.
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.238Z: Expanding CoGroupByKey operations into optimizable parts.
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.280Z: Expanding GroupByKey operations into optimizable parts.
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.307Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.380Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.411Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.443Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 24, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.475Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 24, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.827Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:34.907Z: Starting 5 workers in us-central1-a...
May 24, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T06:45:45.055Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 24, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:56.975Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 24, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:45:57.014Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 24, 2020 6:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:02.380Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 24, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:15.239Z: Workers have started successfully.
May 24, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:15.267Z: Workers have started successfully.
May 24, 2020 6:46:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:52.989Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 6:46:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:53.182Z: Cleaning up.
May 24, 2020 6:46:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:46:53.273Z: Stopping worker pool...
May 24, 2020 6:47:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:47:37.277Z: Autoscaling: Resized worker pool from 5 to 0.
May 24, 2020 6:47:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T06:47:37.320Z: Worker pool stopped.
May 24, 2020 6:47:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-23_23_45_26-1649232473479101083 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 224efef0-f811-4ab5-8830-15a0724d0b13 and timestamp: 2020-05-24T06:47:42.413000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.715
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 24, 2020 6:47:42 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 2 mins 34.002 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 23s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/hbr4vqbtu65iu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/540/display/redirect>
Changes:
------------------------------------------
[...truncated 345.49 KB...]
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a981181e-8ed6-42e5-8a60-8776e643ebc4.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.10/7ca2e4276f4ef95e4db725a8cd4a1d1e7585b9e5/httpclient-4.5.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3f3b986e-8b4b-415f-b59f-ae9eb5dcb5a3.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2524dd7-b4d1-4ed9-87ba-f8cdb4ea80ad.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba9a252a-4421-4655-98a3-822882e5ad6a.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c079b76-ce6e-4bd0-af8a-da12830a9f8e.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06a78a36-e79d-47dc-80ba-0714eb27ffd6.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3f55642e-78a3-461e-b7be-73074929d1d3.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70d58e40-e679-466d-a594-edec4337d2c9.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e09bbe40-9ea3-4c95-b66e-2b0b2cc74dc6.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/796c99d5-b2c8-4392-a242-5c5910f71935.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ade5d32c-9950-40f5-befc-c151c25cbb8f.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb51c88c-dfc6-4b56-b092-48848e0d85ed.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ebfe0766-c6eb-4d74-910d-f2f4e080977e.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a21de285-c450-4e7c-9b88-ce7f30678539.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/894d5fc0-fe0e-4af1-ab99-4598a6f38a04.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed63f17a-3c7f-4a31-a538-7ed241843501.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8bd582c1-9221-496b-92e5-992b576e8c50.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4ae885a2-bd4a-4794-8095-f5c5b37eed59.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a15c61b1-a93a-470f-ba60-0292a40b7c38.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/79e93755-f74a-43a5-b6b7-f7dd5dfa8390.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00b6e4f4-7c8c-4f04-9c58-dc462c370413.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3379450d-b28d-42d1-b77a-98d7755612f6.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/029628ef-03ad-488a-8851-a4fb1dd60bf9.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5211199-58f8-4cf7-bf91-cc5ac6f68dce.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd6b43fa-dec3-4772-9c62-f84a5c89bc3b.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4db01026-5e95-43b2-a976-d3a71db12e91.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4192cd79-3bad-40fb-92b8-03d47cb0bde9.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73507b9b-9484-4def-bb3d-b0f821f59c16.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1fd1acc1-c90d-496c-92e5-c32f8235888b.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5a1dc61-b985-4a72-b1e1-eb00c688431f.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16907f95-e9a5-46cb-9f5b-75c8acab9909.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/71f7d137-d953-41f1-8c56-23abe0b53f7e.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f534a38d-11c8-4442-a579-b45384747b06.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8bd35ec4-0ece-4918-a794-d60c0b02edf7.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d07bd594-841f-407f-9912-d8e28c598c56.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2843eee5-3d69-4241-8809-e993ac8fe809.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8aa508d4-bfd9-4d00-9d48-a99637df61b5.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91d5f0ce-8e11-4d9c-b0bb-8bac1d12db0b.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4d3d5fd3-5503-4791-8844-4f016e723753.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/984a1339-960e-4e2b-919d-920aaea43697.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b8a84651-f1ef-4808-a762-4d69d1112223.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aebdb7a6-8e17-4533-be45-e04f26b95e75.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef4fb236-e46f-4073-bce4-4b06f8428dd3.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3e1661fe-7ab2-44b0-9d38-f94e46862522.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c37fdb76-754e-4b77-bf2a-6521d1e2a67c.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c7cda331-e931-4cc0-b42e-19c4656d9ebc.jar
May 24, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1507540a-8fab-4bbb-9c54-81024371ff13.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/837a4549-3486-457f-9597-bee2de0e317b.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8565c2e7-06a2-4582-b6b3-bf4252c1df18.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31a38ef3-5a8e-44d4-9bac-37dd97c77118.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7cdcfa0b-077b-4940-8b37-7ca2d77ebfc1.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1ca1f06-2e3b-4b79-82ea-4f4f73e371cf.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ea6812d-2339-4d2c-ad43-02a54570272d.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e35fa7b6-4678-4897-9fec-d45382d6d0df.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ad59edf-1cf3-4028-9ad5-aaff09ccf264.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8254ed09-e7f3-45a9-b7e4-0b6a737ba8e7.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e340c39-7cf9-4064-bc88-efadc96122b0.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b9fca577-43df-4f5d-b401-c6fe10a68616.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a984c45f-72e0-4539-8f1c-35daf0cae4f4.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/46624076-c0c4-4005-ac78-6e2bba6811fd.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ec9b9b98-8dac-48f0-9c97-a26e421485ea.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c5e2b7ed-dcef-4be1-a9eb-d16ec19360a7.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/023e77f1-ed28-4e8d-b8e1-a8dd5793b021.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6097323c-dc6d-4f07-9f70-b92374679920.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88fdf026-c1b2-433b-b192-c764b9dd2a83.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0f89904b-603a-4cdd-b28a-ee5e23b24b8f.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/388c6e2f-86e0-4fb9-8282-d73d7a11dde7.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e318c0f8-a58e-4be5-8586-39911ae84ad3.jar
May 24, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b71ab624-c6d8-4870-951b-1f4fd08a3dfa.jar
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 49f391577acc40d77aacc9f21ebc8123ec295d6387410e13ccacbc7a17b5c58b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-08e93d0b-eed7-4b2e-8b81-2d7f52160915.pb
May 24, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 24, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-23_17_45_23-9237404609024828460?project=apache-beam-testing
May 24, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-23_17_45_23-9237404609024828460
May 24, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-23_17_45_23-9237404609024828460
May 24, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T00:45:23.884Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 24, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:27.641Z: Checking permissions granted to controller Service Account.
May 24, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:31.394Z: Worker configuration: n1-standard-1 in us-central1-a.
May 24, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.098Z: Expanding CoGroupByKey operations into optimizable parts.
May 24, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.142Z: Expanding GroupByKey operations into optimizable parts.
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.175Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.250Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.284Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.321Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.354Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.926Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:45:32.999Z: Starting 5 workers in us-central1-a...
May 24, 2020 12:46:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-24T00:46:04.512Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 24, 2020 12:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:46:10.647Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 24, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:46:27.277Z: Workers have started successfully.
May 24, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:46:27.352Z: Workers have started successfully.
May 24, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:47:05.739Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 24, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:47:05.957Z: Cleaning up.
May 24, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:47:06.048Z: Stopping worker pool...
May 24, 2020 12:48:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:48:55.566Z: Autoscaling: Resized worker pool from 5 to 0.
May 24, 2020 12:48:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-24T00:48:55.610Z: Worker pool stopped.
May 24, 2020 12:49:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-23_17_45_23-9237404609024828460 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f449f63d-7e1d-4f93-9963-befa779079b0 and timestamp: 2020-05-24T00:49:00.896000000Z:
Metric: Value:
read_time 21.309
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 24, 2020 12:49:01 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 54.785 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 43s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/zowjthawh5mci
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #539
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/539/display/redirect>
Changes:
------------------------------------------
[...truncated 345.55 KB...]
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/825df498-40b7-4f16-b7dc-a8f85f8e0b77.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1eaecdb6-9669-4b0e-8c13-fd09092c8026.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de296c81-7562-4941-becc-20a12da965b1.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e9e79009-6281-4419-be9f-d0e8214b3626.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/884a7079-0b73-4577-94ce-1682c42b18d7.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/32c4cdda-aa5e-49d7-ba2c-c1efee2a29d7.jar
May 23, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c76a888-1db0-498f-9ec4-554dc0b81f2f.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ea84032-c615-4e5c-95c1-819734448ab4.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1eafbee-4766-4ccc-b852-619a36a1f13f.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d88a3c70-0ef0-4654-8179-b5a5727be68a.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5494156a-a543-4f0c-9be5-dcdf1dbd6bfc.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0fa72a25-f2be-4b7b-89cc-a221e5f1ed9e.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8d07936f-4d2e-4538-b5af-34c706834131.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c0fe1d9-860d-47cd-aae7-b373839f629d.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6092364-98da-427d-b9bb-fa908557f295.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ab6bc167-7bf4-4aea-933d-032bd680e62b.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c2ecaeaf-838b-48de-8209-5283375e77dd.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/259e5999-4518-4c32-8f75-558afd552d08.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dfb6e261-aca7-4641-bce5-98ab244255cd.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d022890-7673-4308-836a-61351470389f.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c88608b2-f2e7-498f-9cdb-bb83846e9d07.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b65dfd79-36af-4ba0-b9a0-fc2399ff54c7.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/316b59c5-5afc-435c-89ea-829b70de128e.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/30a2092a-f068-46a0-8fa8-fde28bb1a87b.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4bc0a515-12fd-408a-b510-6ef87ddd2c83.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58c37791-2df0-4c79-937f-256727aff0fa.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3d3196d9-cb70-4b11-aa6e-560609ac20b3.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb7c1291-6e71-4645-abfe-4156c5305aa4.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b554edf-2b94-4d49-9c2a-430bf4c53999.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eba65a19-17ca-44a2-9fc5-457c3a8572aa.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bbbe9b4-5688-4acc-845b-a9e39e83addc.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1169bbb1-6730-466e-8ea4-06992d7d1032.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a07dac1-3f98-40ba-a662-2836f9acfa98.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/085cc5d7-3088-4754-babf-8f2227eb95df.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e1e9f5d-9057-4cd0-9bef-1b7ac45acd16.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f8063bbd-fe99-4a1b-8c75-b821cfce5f5f.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8be32e0a-29e1-4af9-b3c6-15c6b370d1be.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc0ec1e5-378a-47ca-9a4b-94619d698574.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eb5f8796-c08b-485e-b4fa-2a78c65ec0a0.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1aa45d91-2918-499e-be06-d02e74931a42.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/650081f0-20d0-4e58-9d01-4108cf79e175.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1b2a8ba-d93b-4458-9f0b-480fb1909d31.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/230024c6-866b-4967-a2e6-8b720a70cd1a.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3bfb208-4373-4f9b-ae36-3fba6bfe26b9.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f545a2c7-4ba7-433f-8814-ae95b2d1300c.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a047c86-4845-45b9-9fb7-13f18a53a5af.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1cf65d1-37fc-4069-87f0-a3d372f4fb1b.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/97061aa0-1201-42e7-b478-e6388ac7fbc9.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/28e4ead3-1c96-4059-a2e9-d46d587d1c73.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73f15b53-15af-4af2-8cb2-a5efe7b35fad.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6b14b3f7-3979-4e7a-a0bb-5ac2275593ce.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f06add38-23b4-45e8-aa43-894cea5a40f8.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f0191c4c-8cda-4986-b8cb-85216f4a8068.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87f93c46-60e3-4665-90a5-e7a27bf2d2f3.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6b293196-03f9-4189-9337-977df9cc6d8c.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8dfda22-b20a-4026-85aa-0c5000a70e27.jar
May 23, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c064ca2-5072-4a35-935b-298fdea6a15c.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/410cd659-030b-4d44-82e2-8633508d1015.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/03b9fb36-92b2-4f3e-973c-12bc7ea046e3.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04e11886-7019-4f12-b146-835099e97a9b.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/318da4b7-daff-4f73-b6a5-bfc5a9ded89a.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8bc74d14-6c00-444e-86ac-059df71d88e8.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bf927ee4-5161-4b60-92c3-e68d243fd069.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c96051d-90c2-4d57-b007-81381c7fe55b.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c1859d7a-8065-47ab-8bcf-cb34bdcc2733.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/658a3913-d531-4021-a4db-174f73b2766f.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9568f287-964d-40b1-ae4c-685f2ce6fb76.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a8dcf0f-173c-45fd-b6d6-285b436e415c.jar
May 23, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/36088d0a-9295-4a85-b65c-3cfc9f376dd5.jar
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 4 seconds
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 23, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 4971d8b28ac56b8b32848bf10aa8168d705908248de1fd12a381394af36e1d18> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-eb3689e9-5dc5-4722-8856-a824d0731e59.pb
May 23, 2020 6:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 23, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-23_11_45_24-4131419225917564497?project=apache-beam-testing
May 23, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-23_11_45_24-4131419225917564497
May 23, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-23_11_45_24-4131419225917564497
May 23, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T18:45:24.104Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 23, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:27.499Z: Checking permissions granted to controller Service Account.
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:32.271Z: Worker configuration: n1-standard-1 in us-central1-a.
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:32.834Z: Expanding CoGroupByKey operations into optimizable parts.
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:32.883Z: Expanding GroupByKey operations into optimizable parts.
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:32.919Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.028Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.054Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.085Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.123Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.550Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:45:33.637Z: Starting 5 workers in us-central1-a...
May 23, 2020 6:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T18:45:54.839Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 23, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:46:05.539Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 23, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:46:26.808Z: Workers have started successfully.
May 23, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:46:26.848Z: Workers have started successfully.
May 23, 2020 6:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:47:01.928Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 6:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:47:02.118Z: Cleaning up.
May 23, 2020 6:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:47:02.190Z: Stopping worker pool...
May 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:48:36.632Z: Autoscaling: Resized worker pool from 5 to 0.
May 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T18:48:36.681Z: Worker pool stopped.
May 23, 2020 6:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-23_11_45_24-4131419225917564497 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 45978261-976b-40cb-b0c4-8c29cb9bc315 and timestamp: 2020-05-23T18:48:43.661000000Z:
Metric: Value:
read_time 17.294
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 23, 2020 6:48:44 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 36.49 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 25s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/5oum5oqbeu7lk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #538
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/538/display/redirect>
Changes:
------------------------------------------
[...truncated 345.04 KB...]
May 23, 2020 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07194110-6746-478b-89f8-0f1f2158259e.jar
May 23, 2020 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0ad05c09-6330-4934-9d2b-04fa8739e1b5.jar
May 23, 2020 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f406ef13-89bf-46c6-89c2-48f82eac07b5.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f777afc-a39a-40c4-aba9-4fb52bb69597.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b68176bc-3a82-47ad-9a20-ba12958a22a4.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b163b160-3f7d-4bf3-afbc-dc59a99c9dca.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b209f57-f59c-439b-bda9-f7eff501554b.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70938329-728d-48b8-b24c-87af62c19e79.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e9e42b91-d7b7-499f-b9f4-ccf295a6f3c3.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4288487b-97cc-4f2d-bd3a-794d494f3a58.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/007f3e5a-a0b1-4de8-874c-8ee4396dd79b.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3ddda92-bd3f-455b-8df7-739749f11638.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7f012a19-d8c4-4c11-97f5-c8c12f884939.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61a6524c-6cd1-4a8e-9ab9-2cc9880c7cc6.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/549188b9-636f-4a88-b6fc-6f8e4196b6ff.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c068408c-bb00-4808-b6db-77e9974c2ab8.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07d661a2-f468-4276-b5a9-42ea731806da.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/574a347b-9e5d-44aa-a72e-741ad164e352.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6746f600-d963-48c5-b25e-24165f27536e.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1388b3af-1312-43bf-9e6d-010b0388e370.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a3ee2a8-78a4-4d5f-8106-a1666a585da9.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb4d9f3a-d8b6-4009-98bd-2df8b84a01e7.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/314d4a05-fe14-473e-80f4-93db0b936942.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3633859-7019-46e6-93fc-083799838f24.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/12e4977b-c0b6-4818-abba-4f06c12e1b99.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba0fb92c-ccb6-42cb-9579-b1bcdd77663b.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/019b10c3-50b5-4b08-8e80-5474956abbfb.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3998630d-da85-4dad-9e3c-147738b024ff.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/19bfd573-8ec7-49a9-b2a9-d4c546b37c87.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0cac18f2-5a2f-4e06-8811-96c85b7d4864.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/51a33a3f-caf3-4183-a7cb-973072d0bac0.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3084ddb5-cf78-4b6e-9522-9d805eacce82.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4103af88-c855-401d-ba90-665ce16e896e.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd4d086c-24c8-4a1e-afb5-fb5055bfb351.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c10dfb5e-940a-4dae-b30f-5e53f83804bf.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ced592eb-c77b-4b7d-b65e-1e6f49668ab7.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f516c5ee-6cbf-4e96-b40f-3e43c003979b.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/02c44fc7-2cae-4e26-99e2-74eb7e7458d2.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5b4026b-086a-453d-994e-44e17a101f7f.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/efe5d2c0-d208-4269-a490-ee6ad6a96e39.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6bf58934-acbd-4363-8b1a-8f425c2978fc.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9f3d92d6-1022-4269-8ba0-767cfd436856.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1f1feb13-a2fa-47c2-b571-de5680fc7963.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d915a89-e92e-46aa-8ec3-f11256c2b4e6.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1612ea3-1293-4e5c-bfef-32601597d0bf.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e39e1850-8dc7-4bd8-ade8-6d39a7a87f80.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08135b47-75fc-4c7c-8fe6-2b0e87d99fd9.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11383921-b2e7-4928-9e20-9517252d8e85.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9da9e0bb-cdb6-4deb-8a21-089212cde3ca.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0d431f88-cc53-4e2a-b42e-fb121f7b6c03.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23a05f4f-4c06-4e95-a135-8f013a52c13a.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/680b310f-12f2-4071-97e5-5f749ac17f0a.jar
May 23, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb5f5b02-3491-490a-b310-4d415b539c05.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df56fa67-c689-401f-b3bc-6e052f3d45f9.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6cb0392c-91fa-490d-bf3d-7f754a10935d.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb5e36ab-ca13-46d7-8ffe-aee175aee983.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de5b9f65-2510-4d64-bba4-25496cdc2776.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1bab5f54-fa0a-498c-a1c3-7ec91235d924.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0c000b95-51e0-4c0e-bf1f-9bef10f12701.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b0f8f59-ab6f-4ca3-9012-afb81329c13b.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d56c87d-8e97-4e6f-be4d-6818f121cc20.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0c0b33a-b881-4645-8534-2c8e7c664380.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c008e19e-3fc2-469c-91d4-27e15efe8fff.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9367841-8bc0-4548-8a30-d9e673a3e196.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/971126db-8adb-4a94-b276-25b1c1da24da.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/757c858a-03c1-4a26-98af-9723b9a14475.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aea17b21-a60d-4e9e-8cb4-03e07312bbf3.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0d8ee84b-810b-43da-a468-c0576304c5b0.jar
May 23, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ae0f2a70-6dbb-4785-9814-47487f2866c1.jar
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 23, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash cafc2e8d3e089d8115586b9ca188302946c08c554cba3816b06ddd05eee04b17> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5b088bb7-d971-46f7-964b-25fe0c40996a.pb
May 23, 2020 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 23, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-23_05_45_25-4754738835514482341?project=apache-beam-testing
May 23, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-23_05_45_25-4754738835514482341
May 23, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-23_05_45_25-4754738835514482341
May 23, 2020 12:45:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T12:45:25.403Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 23, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:28.697Z: Checking permissions granted to controller Service Account.
May 23, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:32.569Z: Worker configuration: n1-standard-1 in us-central1-a.
May 23, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.344Z: Expanding CoGroupByKey operations into optimizable parts.
May 23, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.383Z: Expanding GroupByKey operations into optimizable parts.
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.419Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.484Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.519Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.555Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.589Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:33.949Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:34.033Z: Starting 5 workers in us-central1-a...
May 23, 2020 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:45:57.975Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 23, 2020 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T12:46:07.013Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 23, 2020 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:46:17.642Z: Workers have started successfully.
May 23, 2020 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:46:17.678Z: Workers have started successfully.
May 23, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:46:50.677Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:46:50.886Z: Cleaning up.
May 23, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:46:50.977Z: Stopping worker pool...
May 23, 2020 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:48:26.660Z: Autoscaling: Resized worker pool from 5 to 0.
May 23, 2020 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T12:48:26.701Z: Worker pool stopped.
May 23, 2020 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-23_05_45_25-4754738835514482341 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): daae05ef-a5aa-405f-99cd-5ff723fcf773 and timestamp: 2020-05-23T12:48:33.352000000Z:
Metric: Value:
read_time 13.237
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 23, 2020 12:48:33 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 25.544 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 14s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/zwnq55woci5e4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #537
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/537/display/redirect>
Changes:
------------------------------------------
[...truncated 345.42 KB...]
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.14/3cb1181b2141a7e752f5bdc998b7ef1849f726cf/commons-codec-1.14.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e57cf4cb-8993-4db9-be6d-d04884badd9b.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e15fcfa5-3954-49d0-b272-00610c0d8e40.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9458c4cf-a72f-4c1d-ab9c-4ab3d7af3547.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dab357a7-c384-436e-9036-b7bd55668aa9.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/459254b9-b7dc-48f7-abc3-9b36b70ad850.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e00c9cef-7240-475c-9edd-178bd9586bf3.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/625f27df-5168-4340-949d-9569fe8945ac.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6d32106-88a1-4ec1-92b2-bdc2e18890c4.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/32a58eee-b630-4a6c-abe9-0899fbfa3246.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ae7cf17-6f5f-430f-bd05-fbf12f3b2a95.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d07e7ce-4264-47e6-9130-22b43b51e3a1.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54d8ef06-e776-48a8-9a68-f2defc5dce8b.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3844b85c-d105-4eee-b0b1-4e2b38b97082.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f96b24a-23cf-47b4-b9f2-4e2f8634bf93.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/851fc2b7-9777-4fcd-a039-c47773c21e79.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1bb2ba44-b8d2-48b6-870d-b933500fe17b.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/757f57c6-3dec-41ad-84c8-f1d557610070.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e58741c9-52b9-41c4-90a7-83a4060fbc2a.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c5a3ff6f-5629-4cec-8823-4eca48ec3da9.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/40330f47-42ba-45bc-a3c7-a915c32a1360.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f88c5540-b512-4556-8ab2-c245eba072ad.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4371593d-41c1-49f9-bf32-3d92090b1b66.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/48e77dae-64db-4897-ba3b-8783cd4ef291.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5b31ecee-c734-46d7-8960-f5802d4968a2.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/398b8917-a9ae-47c5-b931-1597985eb2d9.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1857951d-cbd2-4851-a6cb-fa9a8731546b.jar
May 23, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49117c0b-058e-43bf-be60-6aa447574a5a.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a811bdc8-8540-4321-97f0-be4cd7af47cb.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e7592e3d-a1b6-47b9-82c2-b14b59ea3c65.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df81a0c5-1c35-46fd-ae8f-2db1d684069f.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0d17f465-a8cc-4067-b88f-d79dc25076ae.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/36bae125-9ac0-4caf-9793-13a000da9c31.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de932cbb-dc3c-414b-843d-86cc0069efdc.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0dd9d20d-c57c-458a-b0d9-8554246ec10b.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5fb5d052-068a-4896-97dc-30f3549a959a.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cfac2f70-03c3-4b99-852e-d99f432d02ad.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8205c477-c12d-4e8a-8a06-21f676678d2e.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/398d9cd2-4bd7-4ce5-aebc-37933ea1e097.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49882dea-3bc2-49b9-9f09-14ea6e8d7c17.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7fb05484-4a77-434e-bdb4-996664b17893.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f04bbb0b-c389-414b-ba9e-fd83b15d050b.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7cddcb60-1f15-48bf-952a-b7e0beb96e8b.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/99776ad7-8efd-4a2c-9657-35eb03c207cb.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/034b6900-0a0c-49be-bbb1-e5592ee7ce31.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6096ac87-1a24-4e33-b690-f00ba593b180.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ca25bed-e3ee-4754-8a28-fe6432d51d7f.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a6be3a12-2915-4de5-a778-0909d28d9ad7.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/782c7c46-5167-402f-b5de-b147bf7e80f5.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d636bb00-0ebc-4a8e-98ea-9522aaf79658.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0cca22ea-299c-485e-9185-8c21d614f819.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91e0d3cb-e1bb-4740-81fd-bc4098f73b66.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e34f2eb6-aaa4-4464-be34-6e7d2c151bfa.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/adc897a0-f286-4607-a57b-89fb03fe6eae.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/92d5dd40-65de-4265-bd57-9f1315f17338.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8954f96f-c79a-441f-b468-410fa98b16f5.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f175710a-0380-41fb-94da-6f6e9c621dc0.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e10273b8-fad9-476d-9e77-9a56953e04f1.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1f98864b-a432-4b24-b88f-4912912484a5.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b805e5a-fb99-4cc5-b5a4-e38da2169bc7.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b80bd7d9-0414-4ce8-93bb-6a6f9bed61e0.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4f9a251f-4b07-4f7e-9d92-945901d32d55.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1882a84-47ee-47ca-82d0-1e53e6628e2f.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/be38b98d-1a17-4169-8293-37ae6108abf7.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6e41021-0fff-4cf3-8ed4-aea7bd4f5833.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a53c852-58cc-4d43-aba5-17e93a462e7e.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3cbc1db-324a-40de-bc72-435cfe682fd2.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d13c7b20-cc05-4bb7-8cc5-4afc75a5de43.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/afad3181-1825-4e5d-818b-816ceb576d52.jar
May 23, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c42edb41-ef55-410a-b2e4-6249d6fef8f1.jar
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 19aeaa630c4c8537f4ccdf1cf0aea6197160aa635dbdccabbca1ab78f96c13bd> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1677f60f-6446-4d0a-ac45-ae9556117187.pb
May 23, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 23, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-22_23_45_27-13868721569358327989?project=apache-beam-testing
May 23, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-22_23_45_27-13868721569358327989
May 23, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-22_23_45_27-13868721569358327989
May 23, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T06:45:28.001Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 23, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:31.236Z: Checking permissions granted to controller Service Account.
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:35.399Z: Worker configuration: n1-standard-1 in us-central1-a.
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.145Z: Expanding CoGroupByKey operations into optimizable parts.
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.183Z: Expanding GroupByKey operations into optimizable parts.
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.206Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.286Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.313Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.340Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.375Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.727Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:45:36.797Z: Starting 5 workers in us-central1-a...
May 23, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:06.254Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 23, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T06:46:07.127Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 23, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:21.984Z: Workers have started successfully.
May 23, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:22.017Z: Workers have started successfully.
May 23, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:58.986Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:59.268Z: Cleaning up.
May 23, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:46:59.349Z: Stopping worker pool...
May 23, 2020 6:48:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:48:27.639Z: Autoscaling: Resized worker pool from 5 to 0.
May 23, 2020 6:48:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T06:48:27.684Z: Worker pool stopped.
May 23, 2020 6:48:32 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-22_23_45_27-13868721569358327989 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ba007bfd-2f8b-4c85-bdd9-68accb3b9f50 and timestamp: 2020-05-23T06:48:32.815000000Z:
Metric: Value:
read_time 21.344
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 23, 2020 6:48:33 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 22.798 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 13s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/vxikbxbszuvb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #536
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/536/display/redirect>
Changes:
------------------------------------------
[...truncated 348.95 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/733f2d88-419d-46e4-ace1-0fe82b5c6b13.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ffcdac4e-367a-42a2-b73b-f82d3256450f.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/65c9fa9e-9ffd-4737-8bc3-9e9479a2ce41.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/97b15482-c4fd-41b5-825c-f9d0da6ba5c3.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a19fded-0994-4e74-9998-95861fbf3561.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/22e0ede1-7cc6-45de-a79a-6398d1ca26ed.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08bd08cd-1122-4dd0-9967-f34c16f54647.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee73e9ed-c000-480a-b8d4-58ac350b0838.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/26403a0a-3892-493a-bd42-051e95ec5815.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e5319c21-686a-4928-aabb-a162f5e543b6.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d77b179a-b58e-490f-b23c-9f042bfb61d9.jar
May 23, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5036d2c0-fc73-47bb-bb86-b301122d4c64.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/403e8820-91d3-4145-9c4b-6e5611fada27.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a63b00f-0072-42e4-8492-8bfb94fc0c93.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/266a6486-68d0-4d83-9bd2-c6b227abe91a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/71baa879-d99f-453d-b258-e38c05027105.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e33eb17-4dfd-4293-baa3-4d165f343877.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b472f095-fa0b-47ca-8fc3-68953f147da0.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4985556d-023f-425d-9dff-77af0ebfcefc.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c03fee7-6510-4f93-9b11-df3eaafeab7e.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ba8679b1-1c30-47fd-a945-69f6a476f5ed.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1daeda2a-fb7a-4d03-9c8d-d47b3cc3ff59.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9b225fe-b751-46f1-9033-131395440ce5.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/236f7124-ae9a-42d8-8f79-2a06a4c5f978.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/798ab487-afec-444d-b353-c0d085b1418a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33ab2f1d-7552-498b-927d-f7e13496cdf5.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dc4318e3-e696-4f0b-855e-5831e2ad079d.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00128f69-671d-43c1-a65e-b26863410864.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bbd8129-3e29-45ad-a596-d182ab851987.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8dda556e-ad6a-49af-8972-0439125c2386.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa1b8b80-0c4f-43e6-a0f5-a0e8f7a2f859.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bba2b25-5ca0-4d34-9f68-8434d16d396d.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73999549-c314-423d-b416-6ab81530933a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/67ed6858-7abf-4c37-bb77-3abf52c4ba23.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bf6cc33c-05ac-4b04-ac52-435513f1b827.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/52ce1214-f579-4977-9397-95ce6488e09c.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/283f65b1-727d-4310-8dc3-e8a578bca87f.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a613057a-3224-4392-879d-35ccd4620a4a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a06e7a7-accd-444d-8984-2b8edef5e3cd.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/508e3d81-ba5c-4fc5-a7c5-4e1850e53e61.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38d3934a-822a-4e77-92f4-76b91c001984.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d6151ec9-e395-436a-8543-828f7b930d99.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1168987f-6df8-43cc-a045-97cdf8d2bb28.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bad22cf8-dfe1-40a0-8b39-2c6ae156e65f.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dc736269-b5de-43b0-ae3b-3ced3368c8e7.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce49b17b-a565-4d63-a472-fe1e8bc9220f.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3ecbd8b-91fd-4d92-bed2-272c953249c7.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d02a0a84-8cfc-4878-b3aa-fbaf8b4458a5.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a838fa04-68b4-4c14-8243-2fc28a3c09ba.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/316084fd-e60a-414d-abce-889bebddd9c1.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9cc45326-b523-487b-8aa4-b58035a46731.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/540bbce9-c4a3-418f-bbfd-97746094f3a6.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4beeaa47-6713-440e-9ce0-b4435f3be76b.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5d749f88-832a-486b-aca7-74c4f855c055.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a4e22438-6a1e-48db-a27e-f7411c2bb0dd.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/75236597-170f-43e9-ae49-4d312dc906d9.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2f680a40-16ee-4899-aa43-7b653532ca8a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eab29770-6137-4f4b-b618-592b5f4de89a.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91678564-0b87-4931-8eb5-411a664c6f77.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b598b72-9cf4-4a1e-8be7-c35b2d108dd5.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44c1b890-15cc-4691-9b8f-20c0219e38ce.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a95ff575-58af-444f-a303-b493f719e6bf.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87fdbcea-2f57-41b7-bdd1-b43097a55041.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d5d5fb75-fa9f-4df4-860b-76bc7f592b74.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed774890-a775-477e-85d5-36a033fb73ff.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82d21c02-c373-4e65-b910-fd9d0a698351.jar
May 23, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bd2184e-6f34-4816-8191-1bb4bcf5c8d5.jar
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 23, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 3fe128d68235e6d40b7bf5bde2e7d147d06de622cded19a206f7d8474d27f0e0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-f945e8b5-cf0e-4c7b-aeab-b2ba711478d4.pb
May 23, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 23, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-22_17_45_40-9598668857294620199?project=apache-beam-testing
May 23, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-22_17_45_40-9598668857294620199
May 23, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-22_17_45_40-9598668857294620199
May 23, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T00:45:40.346Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 23, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:43.140Z: Checking permissions granted to controller Service Account.
May 23, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:47.210Z: Worker configuration: n1-standard-1 in us-central1-a.
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:47.834Z: Expanding CoGroupByKey operations into optimizable parts.
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:47.874Z: Expanding GroupByKey operations into optimizable parts.
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:47.908Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:47.989Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:48.025Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:48.057Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:48.077Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:48.467Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:45:48.552Z: Starting 5 workers in us-central1-a...
May 23, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:46:15.665Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 23, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:46:15.695Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 23, 2020 12:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-23T00:46:19.762Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 23, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:46:21.061Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 23, 2020 12:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:46:45.884Z: Workers have started successfully.
May 23, 2020 12:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:46:45.915Z: Workers have started successfully.
May 23, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:47:19.595Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 23, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:47:19.808Z: Cleaning up.
May 23, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:47:19.892Z: Stopping worker pool...
May 23, 2020 12:48:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:48:52.840Z: Autoscaling: Resized worker pool from 5 to 0.
May 23, 2020 12:48:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-23T00:48:52.883Z: Worker pool stopped.
May 23, 2020 12:48:59 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-22_17_45_40-9598668857294620199 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d4adab80-2d7a-4387-aa79-6a7533b56595 and timestamp: 2020-05-23T00:48:59.900000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.095
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 23, 2020 12:49:00 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 37.565 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 42s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/63efzd32kumny
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #535
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/535/display/redirect?page=changes>
Changes:
[github] [BEAM-10051] Move closed reader check after sentinel. (#11768)
[github] [BEAM-10056] Fix validation for struct CoGBKs (#11782)
------------------------------------------
[...truncated 345.35 KB...]
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/285313ed-2d77-4b0f-9604-db230ffcf286.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4f557599-712d-439a-bf37-d88df47d93df.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/697e7320-357c-4bd7-affa-95c86b3e3bed.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1cdc70b9-a6eb-416b-adc7-b3d6551ef02c.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b93440cc-0d1e-42be-b07e-aab27cb37532.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e21466b7-6307-414c-8cae-be92a5a7732f.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0cdaae0b-dbf5-4c47-89ed-64ca846d68e4.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e6c28f12-1daa-41e6-9042-8fdc68b22fb6.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/53f2fb2b-ff32-4746-89f5-ad7593d0c82a.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dd20236a-559e-405b-9054-92d446558f74.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4fc836d7-2d30-437a-ad8f-926f90c6a559.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/99218e6f-1994-4621-834a-246e8aab81b4.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2649d29c-b25e-4aa6-8319-b9df937ee6c3.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bffd7fe-b0f3-4c20-8b2b-4e7a98e467bb.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ec6afc3e-642c-48d4-bf31-d17205284741.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e419b132-181c-4fd2-84dc-8b4a97373a20.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c27f8049-737f-489c-995c-d0fee2572238.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e73ce9d-ec80-4175-9e9e-0b5b0dd05080.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e6591170-5f8f-455e-89fd-68e3dba6707c.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f746ca2f-948a-4f77-83c0-c47f814eda60.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/15d6571a-6d15-427d-9ab4-39c24ea08fd1.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87451249-7b4f-434a-8523-f98b1a0dc28b.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c21c62a0-690a-4421-adf2-561c74c70d4e.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/664f73cd-2583-4aae-931f-d0923519fcf6.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/080fbefb-d48b-4766-935e-a89c30076dc9.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3331518-c051-4d5d-8884-f104a0ea798a.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2ec8578-7c2f-437c-8254-652b9a3d2763.jar
May 22, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1062f51f-58fd-4318-b584-c31f1d259667.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7ed7dd2f-c20e-4ba9-b158-8db768a0e6b8.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1bd21524-6a8f-4425-bd91-42f5b0f1f8af.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2270f22f-c5a7-4831-8773-a80bae442ba4.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e456e7b-b255-4784-8bfe-29d2b113e808.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5d9845ee-ea91-444b-b634-1e4bf82a90ae.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5b2a9404-f3ca-4cb5-a1f8-08ea908cccc9.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f2e6762a-8cd4-49ab-8aa7-02512cccaae3.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7abf48b3-aa35-4559-941f-cad18fed3377.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/51d9ce58-bae5-4383-939e-c4f7a445d9b2.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/78db7f68-bfe4-427d-89d9-1c7a3a50b1da.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e43225b6-8650-4c56-992b-472dfaa6857a.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d5fc0b8-3818-4d34-be3f-008bc934e72b.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/305a28a5-5d55-4909-9ffc-19431fb63817.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bf2086f1-562a-4ec7-854d-07ead79b7f68.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0a993b6-1899-479d-b65d-cfe43b667daf.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/29bc1b64-dc14-4bf9-ae0b-94151715e570.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5ab5bb05-eda0-451d-b91e-dac8150e179a.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4302d76e-172e-4968-9118-ef3c368147ed.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82559dd6-5896-47f0-85ef-5d48990f5647.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d684f4ff-f300-49ca-ab3e-5013eceee116.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2840a2a4-9a11-47bf-9e6f-264e0c129257.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/10b40855-6fd6-4504-bfa2-849df4021f19.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/715d88a5-30b6-4fd2-8c1a-c9e0e02f9a52.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/37d02122-d50a-4ed3-9f1d-838835f81b9f.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa434b49-8758-47b1-8890-28d37e368768.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce11395d-77e4-4832-b423-6548b9d2af17.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2f0a482f-2052-47c1-92f6-4bfbc1fc4d5d.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73efb27a-507e-4894-b073-b0c2d28b8f59.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3496023-cacb-4556-9790-0d77f641d219.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff6c0f5d-0237-468a-86c6-248f7b76e167.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1ad5e8b-c556-43d7-bfb5-d1357f7aa559.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/75b8efd4-c3dc-43d4-9009-3224980491d8.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74e38e96-e72b-402e-a6b5-1d616076c541.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11776326-27eb-4e38-ba37-00ff454eaa67.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/84430c91-9084-474e-9bf1-47923d3234c4.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/32c4c7e6-b776-4dac-899e-9ba3e21190a0.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8d0ade1e-74ac-46fd-9410-eea1af778f1e.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8fc4365b-e016-4f33-82a1-61a9f1ecc695.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb02e2fe-be0f-4f90-a43a-eac45dc62c79.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85fa1800-5ded-487f-99e1-2801ece56833.jar
May 22, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7d89a474-47f9-4a3a-8825-582eea707149.jar
May 22, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 6 seconds
May 22, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 22, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 22, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 22, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 22, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 22, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84252 bytes, hash aec10ac1f224bf34f5b385ff53eaa08f251703a60141b63982cbb1a805ecb4a1> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2cd45926-686a-4cfe-ad81-9971530e389f.pb
May 22, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 22, 2020 6:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-22_11_45_23-5299817933723028396?project=apache-beam-testing
May 22, 2020 6:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-22_11_45_23-5299817933723028396
May 22, 2020 6:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-22_11_45_23-5299817933723028396
May 22, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T18:45:23.743Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 22, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:27.065Z: Checking permissions granted to controller Service Account.
May 22, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:31.764Z: Worker configuration: n1-standard-1 in us-central1-f.
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.443Z: Expanding CoGroupByKey operations into optimizable parts.
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.489Z: Expanding GroupByKey operations into optimizable parts.
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.523Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.619Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.664Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.731Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:32.795Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:33.318Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:33.408Z: Starting 5 workers in us-central1-f...
May 22, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T18:45:42.921Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 22, 2020 6:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:45:58.323Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 22, 2020 6:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:46:15.850Z: Workers have started successfully.
May 22, 2020 6:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:46:15.886Z: Workers have started successfully.
May 22, 2020 6:46:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:46:49.128Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 6:46:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:46:49.419Z: Cleaning up.
May 22, 2020 6:46:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:46:49.543Z: Stopping worker pool...
May 22, 2020 6:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:48:34.261Z: Autoscaling: Resized worker pool from 5 to 0.
May 22, 2020 6:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T18:48:34.321Z: Worker pool stopped.
May 22, 2020 6:48:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-22_11_45_23-5299817933723028396 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 40728ca8-948a-4a5d-a286-d291f063b808 and timestamp: 2020-05-22T18:48:40.930000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.822
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 22, 2020 6:48:41 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.002 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.005 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 38.007 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 20s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/mcr73uvmmymui
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #534
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/534/display/redirect>
Changes:
------------------------------------------
[...truncated 348.13 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c3ae9978-6845-4164-a43d-b203ac308032.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07b74d59-1727-4609-9291-d2bb43215179.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b07f7ca9-96c5-4aa7-857e-aadc178e5caf.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00d9b23a-37c6-4ddd-843e-226b5d453d77.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c884c7f-eef5-49ba-852a-6e62f6b7c2fa.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49619ab9-6afa-4bb7-b8cb-d8db230c0f87.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a816f8c1-abbd-4219-abc1-9cb1fff327d6.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8bf1aecf-ea64-41f2-bb5c-730667439a65.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1b264eb9-fb46-4fc2-8cc7-8cefb946b9a3.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/325e2b50-2164-4109-81cc-b149c1fd43e7.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/09b8b66b-caee-43f6-a8b8-d2d56d466a04.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c323fec0-8c64-42e3-934a-d34594054268.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/83b96d15-0885-43d7-97bd-e8c9cc9defc8.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3bf37cb-a542-4600-b8d7-721b6f2326a5.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cfefff3f-4e00-4f00-ab8b-f9ad787c9ecb.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2de42a44-ed45-40d1-be96-7434813f20a0.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/53add863-213b-4b3c-a33e-837339aae085.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2c442918-875a-487c-b2e9-4043796f7a53.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c3d00aa2-1bb8-4022-9629-43a92502e123.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/481634bf-5794-4ce6-8fde-8916814ca02c.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23604b74-260b-465e-a463-e5b647a5ed51.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d945261-6f17-4ae9-93bb-422ea5f859a8.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4da09e41-7ae0-4450-a456-cf27a2c1f2b5.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/646aa105-93d9-4797-a8cc-40db98f93475.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/731e5b06-f048-4292-971b-771d18e9ca5d.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/39f5de5f-c969-4bdc-8de8-1bc06193bed6.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d2f5cbb8-2e77-41c9-b05b-57cb51db53f2.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d92463a8-7d43-42d4-9574-918f5f584945.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a222747d-6e66-4f49-9250-1f52591cc051.jar
May 22, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c2c3114-72f5-483c-955e-3f78fe696535.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58f33c14-97d9-451f-99af-ed0fdbe6cfad.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6eb00859-661c-4269-ba24-8d0624928930.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c671b083-b8d3-4500-98c4-97107ea55c6e.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/706dc8df-4b4d-41e8-999b-7234587bce9e.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5b45ffcc-070d-4293-b37d-2d5c09f7cfec.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3cdc6c19-c9c7-4d79-be45-dfd6f111f9ec.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6bdef27e-f75a-4be9-910e-55e9791ed48f.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35ce607d-71d6-4161-8677-975d2d3278b1.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6360c5bb-29aa-4fa6-bf69-182081a6e564.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6def8ceb-0951-4777-928c-8a3967b93a73.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a20a0406-4a25-411f-b767-3a5796f61517.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ef82da7-f5f8-4604-a490-975ddcb0fab2.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b7b9b61c-0a29-4eee-a41a-450f3ffa9230.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3cba69d6-de6c-4911-a9b9-6df596875990.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c12187f3-2822-43a7-9449-f29a41863989.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1cbf61c6-fa0a-4abc-b0dd-7462f34a5090.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/84a03814-7986-4a6e-8371-3349278c42c9.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b0c21b4-b72c-456c-8c20-707e79b73388.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa9edc51-1bc4-4382-9238-cb14d8ebec18.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38404391-5b3d-449d-b811-0e65cbb91247.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c1833ed3-64e5-4978-a7a8-fcaf4fac0601.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a4aaa8c-d1dc-4290-8378-c4e721ff9fb7.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c1890dac-6ccc-4115-8b61-c5661dba26e2.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/943b2d52-a46c-49a7-8ec8-6325f13d683a.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1e02285b-fd82-49de-947e-20ca349d92f1.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc697fd7-8546-4316-949a-01f431a1c70c.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a5e687f-7277-49fb-9b97-03c1a70c4ec6.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2904a562-cf70-4edb-bc92-51f1b2be5871.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2ba1789d-aa3c-45d9-86e2-bc5e6b472127.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8024f4a-b6a4-4cc3-b017-ae3a007381ef.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/26bec9c1-d952-4104-9a78-db604402151c.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33bd3c51-6040-4fa0-ba9c-685e51e8fda2.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b019cf89-024d-44d6-88d0-c659044040b8.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/791404cb-8555-4a0e-a16b-321c4f618c59.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b4a467d9-d35c-41e2-be79-90f944b9cd0b.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1846779e-ab32-4dae-a65a-000271969179.jar
May 22, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d54ee3c5-8c42-4741-a72e-7431a8f584e1.jar
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 7 seconds
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 22, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash b631b67eb0c0ae20527bc10c651b529bafd427661eadba47b09b6a11ac93a1a0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ee6f001b-cc42-4040-85c3-f5c4db47aa8b.pb
May 22, 2020 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 22, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-22_05_45_48-10010410965017672595?project=apache-beam-testing
May 22, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-22_05_45_48-10010410965017672595
May 22, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-22_05_45_48-10010410965017672595
May 22, 2020 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T12:45:48.395Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 22, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:52.320Z: Checking permissions granted to controller Service Account.
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:55.954Z: Worker configuration: n1-standard-1 in us-central1-a.
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.652Z: Expanding CoGroupByKey operations into optimizable parts.
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.692Z: Expanding GroupByKey operations into optimizable parts.
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.731Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.816Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.854Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.888Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:56.920Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:57.294Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:45:57.376Z: Starting 5 workers in us-central1-a...
May 22, 2020 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T12:46:09.346Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 22, 2020 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:46:24.742Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 22, 2020 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:46:24.802Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 22, 2020 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:46:30.251Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 22, 2020 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:46:45.127Z: Workers have started successfully.
May 22, 2020 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:46:45.161Z: Workers have started successfully.
May 22, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:47:20.857Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:47:21.081Z: Cleaning up.
May 22, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:47:21.158Z: Stopping worker pool...
May 22, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:48:57.633Z: Autoscaling: Resized worker pool from 5 to 0.
May 22, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T12:48:57.681Z: Worker pool stopped.
May 22, 2020 12:49:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-22_05_45_48-10010410965017672595 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): df6e6495-55f5-4456-be4a-cb3dcc7ae239 and timestamp: 2020-05-22T12:49:03.135000000Z:
Metric: Value:
read_time 18.479
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 22, 2020 12:49:03 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 34.889 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 44s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ffwdz6qxdfbo4
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #533
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/533/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-9978] Adding functionality and tests to Go offset range tracker.
------------------------------------------
[...truncated 347.78 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a04247f1-937f-4a74-8753-ee65694fb8f5.jar
May 22, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a593ca5-28ee-4086-b282-203c0930d562.jar
May 22, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d850916e-a7aa-4467-9490-44022d8318c7.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45f5daab-8c58-4b5b-a31d-a6e82f8ded9c.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9293375-c702-454e-9709-3c013e8b7dbe.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8204a1d1-3798-43e0-bba1-71b4838ae5b1.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3b1d8ce7-4381-4aa0-8c0e-f2937030683d.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/741b08f9-e75a-42bd-8cbe-0ab2e7888e22.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5433fdc7-ba44-4d97-b302-5c197fed3d93.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/80cda037-7270-467c-aa70-8e4799624185.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4307b637-ad59-48f4-983d-d6d9a5fb22cf.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f60955f7-bd52-41fa-bf23-f9661cf5372f.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ab852f0b-bab9-408b-b508-159729298dd6.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06ed0f96-c426-4152-af71-d06bcec4cc74.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/017dc99b-f41f-4df6-a67e-266fe987f8eb.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/895b2bc7-1227-4a9b-87a1-80c79c56afec.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f50cb741-3e0d-4767-8734-b1f67b75952b.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff2b824e-09d9-4c57-afb1-382743b02cd1.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b14f1b97-3159-4e0e-9973-068a80f6c9e4.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/95b3b5a3-ec22-432c-88c0-4b733ad0c3b4.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e8f5dd7c-8857-42be-82d7-c3799d3bb53a.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c8b407b4-861a-4f87-bf86-c7cc072df86c.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/063414b9-7255-4ef8-bb1e-cdc5734e7065.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4acbfba4-7de3-43a4-9a7c-a97297b12293.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d6f5443b-43bd-4774-9a64-49eb2e2151ce.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/82db84fc-212c-42cb-8103-e32b13b8c5c3.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d38aa156-ff10-44a7-a8f9-a7d55d6cefde.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edfb0f98-c163-4b76-96fc-a0abb80bfb17.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b63ea213-5eac-4f24-8e7f-70326a22f155.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88922f60-c8d0-4013-9e24-690c71cea091.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b8f6a13c-e3ee-4c4d-931d-e5f84ec2080d.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5954d508-df58-4c12-9946-26def07ae691.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1cb857ed-a7a5-45c0-8ade-a8d4ad236cb3.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c0f06383-9554-465c-bca3-f85d77643582.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/032829f3-0874-492c-ad7f-9d50fab92db5.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f786cd46-b0ed-4191-872c-571628f37517.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0830fb17-f6d0-45c8-8df6-cbcd2ced878a.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9b885aa3-7e23-4d20-b045-b347248390b5.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/73fa6f8c-0131-48d9-a264-b4b38d534d14.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f58cbd3a-fa4d-4dc0-89b3-15bdb0904cb4.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/888a7251-9e2d-403d-b7de-277cbb4b7eac.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/191332e7-d825-4689-9b4d-c0fafdc22921.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b43fecb-75f4-41a8-be65-16a19e2a3f7a.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11108b71-5ecf-4a8e-949e-0ad5f9c6fa3d.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/41b2b37a-d4a2-4b2f-af61-508d6c5130f9.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/83838ff9-3073-457c-878f-48cda6a6b7b2.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1ac8b733-30f0-4ed7-8537-d70318a519c1.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c1571ea4-54aa-4a69-8c89-5d5381d0af2a.jar
May 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dc57d3cd-ae57-4eac-81bc-4e0c87e379f9.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f701827e-264f-46fc-97c2-92be8b5f8daa.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87026044-1856-4cce-bdd4-62a4d8280ab0.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91629cda-6499-4dab-a6cb-b8f36fa40b17.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4d6c94a1-f4b8-4876-a932-710c79385d60.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0871de4d-c7ce-4fa3-abc2-dafdc3b5eefc.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8316e8b8-52fa-4147-bcde-45b7303852f9.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b07de48-32fa-4f2c-8516-903b2ff7e58b.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54604068-a5ea-4608-ac88-947e975ffd61.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4446e8bc-77c8-4796-8860-5b7e054eb629.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1053b5a3-d4e6-4a90-b2af-0effdbce0fc0.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bf8590ef-86a7-42af-a4f4-ba46be6d74f9.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a6d167d-9a68-4eb0-8734-9d4410cbcf64.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2244e132-d6b2-47ec-becd-ef73e4422210.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f29179c-21ee-40cd-860e-7d4883096803.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a04f749b-f35e-4783-a623-971ecff3cd2a.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eb728c70-ba10-4284-a0e6-c64fdef6b68c.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f8daa87b-9d95-460c-a2d5-2575dc3ef42c.jar
May 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/186ee957-920d-4ef5-b083-cd3fe7195d22.jar
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 22, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 22, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84255 bytes, hash 319bbe83735b8a37015313126a956e99b7c456f263e2e2c741a6ff3f1b7747cb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3a2bab55-3307-40d3-981e-80d020d1835d.pb
May 22, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 22, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-21_23_45_40-3790058947891932355?project=apache-beam-testing
May 22, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-21_23_45_40-3790058947891932355
May 22, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-21_23_45_40-3790058947891932355
May 22, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T06:45:40.748Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 22, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:43.847Z: Checking permissions granted to controller Service Account.
May 22, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.037Z: Worker configuration: n1-standard-1 in us-central1-a.
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.725Z: Expanding CoGroupByKey operations into optimizable parts.
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.757Z: Expanding GroupByKey operations into optimizable parts.
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.789Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.868Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.899Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.920Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:48.952Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:49.259Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:45:49.331Z: Starting 5 workers in us-central1-a...
May 22, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T06:46:11.464Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 22, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:46:13.331Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 22, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:46:13.350Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 22, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:46:18.747Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 22, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:46:33.410Z: Workers have started successfully.
May 22, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:46:33.435Z: Workers have started successfully.
May 22, 2020 6:47:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:47:05.254Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 6:47:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:47:05.419Z: Cleaning up.
May 22, 2020 6:47:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:47:05.478Z: Stopping worker pool...
May 22, 2020 6:48:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:48:33.718Z: Autoscaling: Resized worker pool from 5 to 0.
May 22, 2020 6:48:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T06:48:33.755Z: Worker pool stopped.
May 22, 2020 6:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-21_23_45_40-3790058947891932355 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d522fdac-7b65-452a-ad34-e32fd223e8f4 and timestamp: 2020-05-22T06:48:40.337000000Z:
Metric: Value:
read_time 13.546
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 22, 2020 6:48:40 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 17.813 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 22s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/6fj5ghnzvdjvg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #532
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/532/display/redirect?page=changes>
Changes:
[heejong] [BEAM-10052] check hash and avoid duplicated artifacts
[github] Website build: Remove -u root (#11625)
[ankurgoenka] [BEAM-10060] Upgrading Python containers to beam-master-20200521
[github] [BEAM-9722] added SnowflakeIO with Read operation (#11360)
------------------------------------------
[...truncated 348.06 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.14/3cb1181b2141a7e752f5bdc998b7ef1849f726cf/commons-codec-1.14.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6b9843b2-caa3-452e-a96b-199fc9d2b468.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/60acb841-2829-4062-85fb-a0b1dc21a186.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6ef3ffd7-3fd4-4acb-b4fd-1f63aa1be9ef.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14f03599-b248-448a-9907-619810d8889e.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b98f9da-da4f-4d93-985d-101204939c5a.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f070c727-0982-4b8d-b12e-d953fac98da2.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e235da42-5f2d-4690-a46c-61798a0ad6ff.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/26f85e29-01c0-40dc-a3a5-4155bf318fb5.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b634331-d567-410f-9696-6f8790d018fd.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70f6f603-7aa6-4dab-aedb-728f828d5818.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/71861830-a178-4160-8547-8625938a05ce.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4e6e1081-fe4b-480d-816b-4a67fc34597e.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee5e01d4-b2ca-4c56-bcfb-9d05085e321f.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/15ceab53-68d9-4956-8fd3-cc0f50110409.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44c304f4-d65b-4508-a1cb-15aae4d35ed2.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9750818-024a-4092-8326-0a4cc6d58a4c.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c5b721a8-98b1-4408-b69d-bed33c1ebd5c.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/452c9d2b-6057-4098-8788-4502eb2d3050.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a82fe4c-7697-4c8a-91a4-c92faca23d12.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/505197c4-acfd-4eb2-9ac6-3abf6a7e36e3.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/908772ee-5d6b-4be4-92eb-0e400d48e61f.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bef530a-e1bf-486f-84bc-bcbc9490f814.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a7205fc-b886-4c5c-9259-125b4e265bdd.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e442e8a-74be-4dd2-8a31-d4aa926de419.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/834a308b-648d-42f7-b4bb-7381683631b3.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1b05c57-f750-4121-b8ba-ee3f50045b3d.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0749f1ed-acf1-4b3e-9c17-8723d54830ca.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b9bcf6e7-686c-4d5a-a695-a07130dd794e.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c679fcd8-9c7f-4c5f-bf21-3b72950bc739.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0d8b6206-1fbe-4903-84b6-d9571c0e2a3a.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08ab88f0-c0e7-474b-900b-c14b4f74beaf.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ada449a7-0682-4029-8825-15108744b1b8.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/05beafed-d586-42f5-88ae-5c51316cb6de.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6929f95-c44a-479b-9998-f3976a2050b1.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe73df1f-181c-4734-8edc-002cdda8befe.jar
May 22, 2020 1:43:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1327f147-21f9-455c-85e1-e93788af21c8.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/90272ffe-1ebb-4f57-999f-e783e997f65e.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6a99393a-18e2-4ab8-8c07-7898707c6d9f.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0694b94f-71f8-4209-b131-0bb2714260e6.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e43bd251-91f0-4e2f-b1a9-74f486209f5a.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3aa0ec09-f2be-44e9-8d90-3ce0c99ddb77.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1a3d0ff-1ec1-4b9e-ae9d-bf7ffd7b7c40.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/726b3f93-a12b-42f0-b856-e6cc112b5fdf.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/133a654a-47a0-4f9a-a24f-27699b9acbf6.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bc9936bf-9db1-4535-877e-9766846abbe5.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/266d0cb3-1179-4e15-ad1e-2a16e2aa3911.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dd1fedbf-42ea-4fb3-ae63-6a19c98ceac8.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/da9797d1-b28d-445e-9464-ea5ebcc33655.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/791207fd-06ac-449b-9f17-c82c47076aea.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d24bc45e-aac8-439b-8117-45d18443a74f.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dcd852a7-657b-42fe-b982-826158e821b8.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bb315e4-635f-4075-bef5-cb609f5ad652.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8c2a3874-8c0e-4b2f-82f6-f7790b4b151d.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/256e6664-c56e-4d6b-b884-87113c48c431.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fddd464f-d110-426a-a122-435db2029be9.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/86ba46ba-b9b7-4323-8582-db89b667bce8.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/347249d5-7643-43df-b27f-2b3b2ecb8e56.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d500021-952a-4ba4-93a1-e10f4599bea7.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6f683d1-8960-47f1-b20d-14239eeb5866.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/92ad2ba9-49f4-4f4d-b0f3-bfc024974b5f.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/20a5730d-9453-4982-95cb-99464a91d6f1.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/853d3ade-b7a0-4eb6-b909-2188003f3cf8.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/abde91f8-f65c-4361-a978-4a11d081804d.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2b3296ad-7a21-41a2-be0d-3547c3299ebb.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c66200f4-8788-497d-9317-0f5693da882d.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0037f262-5757-4ad8-8db2-c1ad46729df3.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7d92e7f4-e6e2-48b1-91ec-7525cbed1343.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8aa7a778-f024-49d3-b420-058f97183209.jar
May 22, 2020 1:43:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e18f5319-3abb-4024-b7ef-211ef61e2208.jar
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 6 seconds
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 22, 2020 1:43:52 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 8e89a37508affedcbd3f253615c6482285a4900c7e3e8a1e8eac72b4d2b382db> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-461ea55d-9dfc-423a-b096-dadea077a6dd.pb
May 22, 2020 1:43:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 22, 2020 1:43:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-21_18_43_53-13951273707026062289?project=apache-beam-testing
May 22, 2020 1:43:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-21_18_43_53-13951273707026062289
May 22, 2020 1:43:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-21_18_43_53-13951273707026062289
May 22, 2020 1:43:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T01:43:53.380Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 22, 2020 1:43:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:43:56.556Z: Checking permissions granted to controller Service Account.
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:01.163Z: Worker configuration: n1-standard-1 in us-central1-a.
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.071Z: Expanding CoGroupByKey operations into optimizable parts.
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.150Z: Expanding GroupByKey operations into optimizable parts.
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.288Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.464Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.509Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.549Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.588Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:02.974Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 1:44:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:03.045Z: Starting 5 workers in us-central1-a...
May 22, 2020 1:44:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-22T01:44:12.707Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 22, 2020 1:44:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:27.655Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 22, 2020 1:44:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:44.867Z: Workers have started successfully.
May 22, 2020 1:44:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:44:44.909Z: Workers have started successfully.
May 22, 2020 1:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:45:19.033Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 22, 2020 1:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:45:19.260Z: Cleaning up.
May 22, 2020 1:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:45:19.352Z: Stopping worker pool...
May 22, 2020 1:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:47:04.010Z: Autoscaling: Resized worker pool from 5 to 0.
May 22, 2020 1:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-22T01:47:04.056Z: Worker pool stopped.
May 22, 2020 1:47:10 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-21_18_43_53-13951273707026062289 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 49b0a703-763b-40cd-a474-84c886e66973 and timestamp: 2020-05-22T01:47:10.767000000Z:
Metric: Value:
read_time 15.919
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 22, 2020 1:47:11 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.04 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 38.677 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 50s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/psxjgiqysdb7e
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #531
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/531/display/redirect?page=changes>
Changes:
[rohde.samuel] Make GroupByKey a primitive
[rohde.samuel] Move ReifyWindows etc to DirectRunner, don't subclass from GBK
[rohde.samuel] remove references to _GroupByKeyOnly
[rohde.samuel] add back the ReifyWindows to the GBK, some internal tests depend on it.
[chamikara] Clarifies Dataflow execution environment model
[chamikara] Fixes typos.
[rohde.samuel] Skip checking input determinism for GBKs embedded in x-lang transforms.
[kcweaver] [BEAM-10048] Clean up release guide.
[kcweaver] Remove checklist from build_release_candidate.sh.
[chamikara] Clarifies text.
[rehmanmuradali0] Update changes.md
[github] Include additional instructions for staging repository.
[github] Clarify the usage of sign_hash_python_wheels.sh.
------------------------------------------
[...truncated 348.12 KB...]
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ad0f405-2145-4707-8369-1ceeb26f8182.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0d7fd88f-51c9-4ab8-8183-8164b573b73d.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a61db367-c4f5-47d5-99bb-f52a9c3d6e4d.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e9b09b91-1300-43d9-bda1-034e79190296.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31b64d94-6b97-4930-98cd-a07d87450f1b.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27059b4f-43f4-4d89-968a-f0991539b11c.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d5551016-ef8a-4bc4-9c9c-e085b9ac650f.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f44a515c-0dc1-4621-9866-8ecbfa060a92.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07b56ff1-61ff-49aa-93bd-6f1e02bb136e.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/79ac832f-7ef8-467b-a564-c48666d975d9.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5e96262a-ddef-46ee-b7ff-5313f5098406.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/80cdf478-9110-42af-b63e-2698d0cad656.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d6e34c66-6b16-4f6e-88d5-cad31c6ec4d0.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b92a9adc-d426-4fca-9790-c21f166d7088.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bd2b82f3-2fb1-42ba-b4e2-e7ab2c823542.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/62a3bbcd-b020-42e7-bd03-9edeeecdec01.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9607ee32-8c3a-4a8f-b385-46c8e28bea97.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5e6f1401-d9e1-4f49-b128-64de0adcda3b.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8c5329b6-b45b-4fc3-b44e-7df5db55834c.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e861aa3-b889-41b9-b33e-38348ba5a19c.jar
May 21, 2020 7:29:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3de7c3ab-7e9b-4e81-9348-a71db60eda77.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3240543e-a54d-4126-8249-d251c02a46c8.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edaf931f-86c2-49b4-92a2-26729446220d.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b876c75-b6d8-4d35-8701-c0a47c0bbabb.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a19ec397-2573-491b-b590-f7750ddefe02.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce48d557-5687-4f21-a6b3-28a1cca58a63.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/66007c63-d6bf-4175-9964-ae353231645c.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aae26956-f4da-49fa-95e9-e70e793e5ded.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2b3d3db7-d315-4ee9-827e-f682f2b4e2cd.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6937015b-ae29-4985-b28c-056ebc40b4f7.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2e4bb63-d2f0-4357-b584-6aeb64648652.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff5dbaba-88f3-4750-8bd9-298640303745.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45a20dee-825b-4a1e-8089-525d1bcef41e.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d33734a2-700c-4a9b-b62d-bbe912e43c8b.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cfd23213-88f4-4e98-b1bc-7ebd552f1bba.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/441187c3-df5b-4929-948f-9ecf6511266f.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5797bd5-4dd8-4553-ab68-04f31951f84e.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bb0dedf6-54b8-4dbb-ae01-81ec6f8788a7.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f17b22b7-cc62-46ed-a25e-2a1b7816879c.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d663bb09-9b62-4adc-9508-cf64eab6cb39.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/376397f8-5c95-49f1-865c-486db490f073.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/354a9b5b-e9b5-4f49-8053-fadec12c567f.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee0d7f7b-a8ab-451c-b775-c765b6bbf2bf.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/145016bd-6be4-4621-b920-81600075d167.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1b05ac5-6df8-44dd-bf6b-cba093fb209c.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a030e652-e7ef-4d25-b84f-2e07ca735613.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6f232fe2-4a7b-45ed-88cb-0e800ac52609.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/acf71437-740b-441b-9eb2-99cbce6dc32b.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bea21df4-a87c-48df-ad4e-7d2d130cfabd.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3afc5033-76ba-4e5e-9323-e57aa50af517.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3a82582-c88b-471c-a43b-1881316d3c19.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bbc5484b-9780-4fc2-a29d-7a00927c8ec4.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3137460-2a35-4f59-82b3-401c2758a3c9.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1738f03a-df31-46ce-91a2-66cf8b839a1b.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9ab25e13-18c1-4826-938c-aacabc36635d.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/628543ae-e70a-4305-acb1-5b3a6eed3c8c.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6c1d135d-62bd-45e1-8d6b-eaf752fdbad1.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1568592-e4f0-4579-a130-1dc75eac9764.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fb96becd-bacb-43c0-be70-4cca70321adf.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a64bb81b-c37f-49fa-87ac-4c1172e96c2b.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0a6f5b3d-5294-4766-a059-3b605d72a983.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/afde1248-842a-451c-8846-c6697609f603.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/056298aa-cc3c-4a4b-b2ce-32a498f9c659.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/050220f4-bee9-4115-beec-e4bab7b87498.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/05785b4a-ca87-45a9-9b1a-c4a4021be196.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/55a3c619-4ded-4348-a862-c40738bad57b.jar
May 21, 2020 7:29:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e04e5816-cd0c-428d-bd80-9e44093d216e.jar
May 21, 2020 7:29:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 6 seconds
May 21, 2020 7:29:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 21, 2020 7:29:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 21, 2020 7:29:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 21, 2020 7:29:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 21, 2020 7:29:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 21, 2020 7:29:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 67ded7ef7a7ed2691e21b428ae31f50c831cd0738181a530a0009631742ed084> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5b72bf60-aee8-4969-8af2-2f48f21b0a42.pb
May 21, 2020 7:29:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 21, 2020 7:29:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-21_12_29_30-8680741031390412447?project=apache-beam-testing
May 21, 2020 7:29:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-21_12_29_30-8680741031390412447
May 21, 2020 7:29:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-21_12_29_30-8680741031390412447
May 21, 2020 7:29:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T19:29:30.762Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 21, 2020 7:29:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:36.396Z: Checking permissions granted to controller Service Account.
May 21, 2020 7:29:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:41.513Z: Worker configuration: n1-standard-1 in us-central1-a.
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.131Z: Expanding CoGroupByKey operations into optimizable parts.
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.191Z: Expanding GroupByKey operations into optimizable parts.
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.227Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.326Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.359Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.394Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.436Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.837Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 7:29:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:29:42.908Z: Starting 5 workers in us-central1-a...
May 21, 2020 7:30:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T19:30:02.612Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 21, 2020 7:30:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:30:09.360Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 21, 2020 7:30:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:30:09.385Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 21, 2020 7:30:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:30:14.760Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 21, 2020 7:30:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:30:31.123Z: Workers have started successfully.
May 21, 2020 7:30:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:30:31.165Z: Workers have started successfully.
May 21, 2020 7:31:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:31:04.312Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 7:31:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:31:04.533Z: Cleaning up.
May 21, 2020 7:31:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:31:04.627Z: Stopping worker pool...
May 21, 2020 7:33:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:33:11.970Z: Autoscaling: Resized worker pool from 5 to 0.
May 21, 2020 7:33:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T19:33:12.009Z: Worker pool stopped.
May 21, 2020 7:33:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-21_12_29_30-8680741031390412447 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0f73c856-b062-4864-b563-7362efa0f576 and timestamp: 2020-05-21T19:33:18.866000000Z:
Metric: Value:
read_time 16.673
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 21, 2020 7:33:19 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 7.029 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 58s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/4pnykmnwp5mb6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #530
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/530/display/redirect?page=changes>
Changes:
[github] Fix grammar / spelling
------------------------------------------
[...truncated 349.16 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b9fcd214-4934-4410-abde-8446e68c11a5.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eac31fb2-3540-4b56-a2e3-f12ecf18a640.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f3ce2e9f-f1af-4df1-97f0-35e3ca9500c3.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4966d9cb-34a2-48fb-b6cd-13b906bbe46d.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38c1ecdd-2839-4146-a07c-f4adffdc6a53.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cc6ff152-ca3b-4e03-8aa7-cd31c54f6629.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3f1a34f1-cc45-480c-b70d-5160dbd08ab0.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/efa88f13-2c8b-4724-a4e8-2ec0d0d19216.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c2ec2b4-756e-417e-9ec7-22815a241036.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/988fa7fd-7f18-4c5c-b6d9-532aab8ee27c.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/646cc156-f6f1-4e82-a311-b11d171508d3.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e97d7e18-1b52-4dfb-aeb9-1cb3880ff03d.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b75abdd1-6998-43a9-8bd9-1032f312676c.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6957afd-ce1d-4a08-b067-51bd4fecbc50.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ca67bc16-436f-4b63-9be2-53bdc420ce7b.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/00ee92a3-fe0f-41ff-80d9-e955ea42fcc4.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a597e52a-285e-43b5-b90f-2a8e1609531d.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/15df8e93-f37b-4130-b643-71f444b1753e.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88e83615-d6ab-449f-bd92-a0ec8c50357b.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/20789b50-4e5d-4242-a27d-a0da29020399.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7dc06420-f85c-486b-b376-36aa5e056559.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eabf04b4-0c0c-4b4c-ba03-60c536c1b8ff.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb808512-6acf-4f3c-b12f-de128340a04a.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8ef274c-e8b4-4ae0-a2da-95cf37f0e542.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b11dddc-84ee-4e38-ac8e-4193f6991e61.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9db575b4-d28e-4caf-be8a-60d000eb124a.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/446fb199-896f-4202-ba12-dc74134398f9.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5e737a58-e40b-4e2b-bb3a-c1750617c515.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d22ee293-8334-409e-88d0-6c8d8c731c66.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e2a494b8-40f3-44eb-a519-6c3f44cde90f.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/339556a6-de6c-4f76-9c35-6dd5ede39aa0.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a0c475c-5829-4fe0-8b84-5151c54a1aa0.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8e9cde31-fdd3-467a-bbe4-7d33b679478b.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9a9365a-129f-4760-ad58-749fbe4962b3.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3d5dc93-ba93-4695-a411-fc9d08c8763e.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cfbbeabf-c6a5-4a5c-8275-5bab4224297e.jar
May 21, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/553a7908-8acc-489e-a44f-daeb1f030a7c.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/707532bf-ca04-4b60-98a3-618425e7dfbb.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/954a8060-1123-4041-93af-cb24830f2c1a.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0cad81ff-98e9-4e95-b80a-de55e316cdf4.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/45999b46-94f0-43d6-9e3f-b3c37c737ec9.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9789d98d-6852-43a8-bcad-6bac824d0037.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aa11ce91-c934-42d0-bec9-5389b0b46df8.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c34a747b-e810-4368-9ffd-d7ba5b6035fd.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/67fffaa6-c4dc-4b16-9d15-b498d3c8a776.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25523889-f17d-41e2-b4a6-6bd1bd3ec536.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e650a3f9-bb76-48c4-954d-0e6fa97a2cd9.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3baf2ac0-83de-44be-8125-3a0a018833db.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2745a1ab-3f76-4f5f-80c2-9392a84d9da7.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a60cbd5f-359c-471b-b188-ab39a9dfa703.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35a9a784-64b1-414b-a6e6-1c88eb7d0265.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fdcc586b-21d2-4763-bcb1-b00e54b518ff.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/33def5b9-60d7-4a26-932c-450d3f1d30cb.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/96e2ab2c-38b0-4497-91cc-e49d43929df7.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/af2e6b1f-2e2f-4aff-b127-0989f34e0f13.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ac030d12-50c4-4d14-8056-3ac7a636e7d7.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b455707a-00e3-4d86-b8cf-046ffb1a260e.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/282faf7a-babc-46c7-941b-b24cf020ed7b.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5e1f5691-99c6-4da1-862f-404b790e4d39.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/428498c6-b901-480d-8b64-3dafbef1caa9.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7f8fea3f-3147-4841-a3b3-c05039bf399c.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b32ff60e-f6b3-493e-84af-67a145fde4a4.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f7c136ad-2f4c-4729-bfea-0fc07c50e45e.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/456254bd-acbd-4d77-82a9-ac2dd751c5df.jar
May 21, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1df7614-a906-4de6-9617-474f8464a882.jar
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 21, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 21, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash e24976b0ec725d4aaa2f48a4348f52a8369930d1e46e89e9ba3eb2b861fe4096> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-36d9ade7-054d-46c7-b9a5-ed5a38c96cf1.pb
May 21, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 21, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-21_05_45_37-13421834415099248458?project=apache-beam-testing
May 21, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-21_05_45_37-13421834415099248458
May 21, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-21_05_45_37-13421834415099248458
May 21, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T12:45:37.621Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 21, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:40.520Z: Checking permissions granted to controller Service Account.
May 21, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:45.583Z: Worker configuration: n1-standard-1 in us-central1-a.
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.236Z: Expanding CoGroupByKey operations into optimizable parts.
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.275Z: Expanding GroupByKey operations into optimizable parts.
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.299Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.370Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.408Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.432Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.463Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.911Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:45:46.991Z: Starting 5 workers in us-central1-a...
May 21, 2020 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T12:45:59.693Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 21, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:25.936Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 21, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:25.972Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 21, 2020 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:31.315Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 21, 2020 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:31.357Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 21, 2020 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:36.798Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 21, 2020 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:47.235Z: Workers have started successfully.
May 21, 2020 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:46:47.272Z: Workers have started successfully.
May 21, 2020 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:47:29.277Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:47:29.477Z: Cleaning up.
May 21, 2020 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:47:29.546Z: Stopping worker pool...
May 21, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:49:16.648Z: Autoscaling: Resized worker pool from 5 to 0.
May 21, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T12:49:16.693Z: Worker pool stopped.
May 21, 2020 12:49:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-21_05_45_37-13421834415099248458 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): da64e867-5302-47e1-9a69-55bfed6ab42c and timestamp: 2020-05-21T12:49:24.098000000Z:
Metric: Value:
read_time 22.686
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 21, 2020 12:49:24 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 4.033 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 6s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/r3li6mcwu4ofc
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #529
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/529/display/redirect?page=changes>
Changes:
[veblush] Upgrades gcsio to 2.1.3
[veblush] Add grpc-alts and grpc-netty-shaded explicitly
[github] [BEAM-9822] Simplify pipeline when batching is disabled. (#11529)
[bhulette] Moving to 2.23.0-SNAPSHOT on master branch.
[github] [BEAM-10028] Add support for the state backed iterable coder to the Java
------------------------------------------
[...truncated 349.97 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b1c4fe1-fb22-4bfd-b9cb-d338c1be26a9.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1039561d-8a5c-4500-9633-b91a87370e47.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/919b8437-6e77-4d6a-9d17-c7e43f620995.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3574642f-066f-4307-83a4-ceee66d55ab7.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8915c4f4-2dd2-4b2c-982b-6a3e01ee9ceb.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3a4bc382-5578-458b-9e22-2d4f11765dad.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/536ddaeb-4743-4626-96de-317cfa1cea31.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/be437898-b698-48ee-85d0-d8c41fccff5e.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a18ebd80-98df-4b07-986a-e59c57897aa9.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d1125fc-4871-48a8-b187-aadead7e005c.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/289abc84-f62c-4c54-8b1a-e1344b72fe22.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7.1/58d76a9ec581f7c6d33f3343de9b2ba04a0ae799/auto-value-annotations-1.7.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/799d950f-3c39-401e-b61f-62e3ca4676d1.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/50b59087-a3ba-4069-a749-f668d7ec6eb7.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5.1/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a7389e60-a506-4aa9-aacd-1a7eeff8f61e.jar
May 21, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5.1/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/848305f5-8790-4f2b-95df-3d512b71b3fb.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5552286b-44dc-44c1-936d-efcc9acfbad2.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/459cda21-4d09-4c1e-9253-32de0783b0b9.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e192115-a852-45f8-814c-7a51229d2d25.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/91724610-9899-46d6-b3ee-0c262024d63c.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/adac1c86-a46c-40cb-b670-9fdb6eb3abb5.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8b277eb6-9051-4797-97b2-9b0bed621031.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d17ac65-b290-4804-9d43-76d5a7bbfda8.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/01d3b420-302f-4401-aeff-e66a16595dfc.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/34af4991-8a71-450f-ae77-7c8d6f4a53c3.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9dfa48bb-796a-4e09-a715-114006b6ce48.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa01a582-59b1-4f26-803e-056e81664097.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74467ea3-6fe8-4929-b9f2-282e1509b1dd.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ff48efa4-f5d7-480d-91df-b04552eab68a.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e1e59ab-c86a-457e-8a2c-efb02f917403.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9dc60e61-dded-48a0-b5a4-15f40184782b.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd91fe6e-0c52-4cce-9949-bc781ac882d5.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d85e7fc-3724-490f-b192-691489722514.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16392eb4-20eb-4cf9-9a24-da5f84370fa1.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6a515d1f-0278-4f07-9697-9ba0636827a1.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a1aa6ef-a85a-487a-aab9-111caaf6eb9e.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8699ec01-7b51-4c44-a9b7-b494c953fb19.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3f1b007-2307-479c-a8e7-7b86eb270afc.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9aaf0677-6fc7-4176-8ef3-fe7396177607.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e36577db-6859-42b0-ac2f-d1dd8f5429fb.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e39609c8-befb-4f67-87d8-d9cb48102cbc.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7288f4bf-6aef-41d1-ae4e-3f9a3be4a9e4.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5.1/71d1e2cef9cc604800825583df56b8ef5c053f14/flogger-0.5.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a2fb03e9-58e0-43d1-8769-0d4e35c596c9.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8a3e8f7-8117-42bd-88b9-5c893b365fad.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef6968dc-65ce-4f59-9c44-6affc40f2dc7.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/095b2df5-3a29-45c3-a220-a4ba38ea945c.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6f15cb6f-fbb0-4095-8d7e-525dc5c44a9c.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0bf61f41-32eb-4457-8ea2-60d60814cf42.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e543b51a-01bd-4b57-bff8-72a556760bac.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e11c6075-e243-482d-9f91-60036d5cc121.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/48bbe2ab-8bf7-47c5-8015-3c7012dcd590.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d23e6419-e331-4d3a-8328-42e40b8e26b2.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef10b7bd-da73-4559-a48c-2aa9090d1dc1.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f5e3a840-c30d-4fc6-8b57-d37ea89dde72.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49faef20-e72c-4120-8e02-8b27ccb83ec2.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c086130-7c7c-4017-b59f-b97680195b97.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a3d03009-950c-494a-ae79-f1fbbcab6945.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bc9b22cf-6fc2-46f4-b64e-fbb933cfd595.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/68aa4a98-5420-4bf9-acca-1288646fd7bc.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/635ea5cf-29fc-4725-b94f-c1f2ddc852e7.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ad8ca99a-8b2c-437d-b0d0-199798b6b6cf.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c4e983e-828d-46c0-a2e2-a0c5a2cce2d3.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/21ee637d-1724-46c8-adde-9620378535fb.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ebd2c8a8-46d0-4d81-9d1f-4189e541776a.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edc92e0d-0283-49fb-a219-3b577d379c88.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e50f4632-c9ad-4615-8928-959d772e5344.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c5feb47-39a3-4823-a43f-591b0d493f89.jar
May 21, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/150e783c-e145-4f65-8f7e-c07de2f6b99a.jar
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 21, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84253 bytes, hash 260dc4e0b0359bf942d3c266c8e7a4bf7d91eac1eb2a2328aaab3ac3c1189d0d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-94222fb2-2bf4-43b4-8c32-a70498ffcffb.pb
May 21, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
May 21, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-20_23_45_51-7710337663155172068?project=apache-beam-testing
May 21, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-20_23_45_51-7710337663155172068
May 21, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-20_23_45_51-7710337663155172068
May 21, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T06:45:51.404Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 21, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:54.549Z: Checking permissions granted to controller Service Account.
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.019Z: Worker configuration: n1-standard-1 in us-central1-a.
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.581Z: Expanding CoGroupByKey operations into optimizable parts.
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.619Z: Expanding GroupByKey operations into optimizable parts.
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.646Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.739Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.768Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.797Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:45:59.835Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 21, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:00.301Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:00.378Z: Starting 5 workers in us-central1-a...
May 21, 2020 6:46:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T06:46:06.380Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 21, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:28.012Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 21, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:28.052Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 21, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:33.424Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 21, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:47.650Z: Workers have started successfully.
May 21, 2020 6:46:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:46:47.679Z: Workers have started successfully.
May 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:47:23.004Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 6:47:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:47:23.198Z: Cleaning up.
May 21, 2020 6:47:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:47:23.283Z: Stopping worker pool...
May 21, 2020 6:49:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:49:37.297Z: Autoscaling: Resized worker pool from 5 to 0.
May 21, 2020 6:49:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T06:49:37.385Z: Worker pool stopped.
May 21, 2020 6:49:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-20_23_45_51-7710337663155172068 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 30d4b141-1d72-4140-b609-fd9814608ca3 and timestamp: 2020-05-21T06:49:43.814000000Z:
Metric: Value:
read_time 17.139
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 21, 2020 6:49:44 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 11.445 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 25s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/wcy3ppvg27hnm
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #528
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/528/display/redirect?page=changes>
Changes:
[github] Updated Videos and Podcasts page
[kcweaver] [BEAM-10038] Add script to mass-comment Jenkins triggers on PR
[kcweaver] Add mass comment script to release guide.
[aramaswamis] Remove curly quotes from documentation code samples
[kcweaver] Update test list.
[kcweaver] Update verify_release_build.sh to use mass_comment.py.
[kcweaver] Run yapf on mass_comment.py.
[github] Remove irrelevant instruction.
[github] [BEAM-9679] Add Core Transforms section / GroupByKey lesson to the Go
[github] [BEAM-9699] Add test verifying we can use ZetaSQL in Python SqlTransform
[github] [BEAM-10037] BeamSqlExample.java fails to build (#11754)
[boyuanz] Implement GrowableOffsetRangeTracker
[github] [BEAM-9899] Fix some issues around storing schema `id` on user types
[github] Merge pull request #11339 from [BEAM-9468] Fhir io
------------------------------------------
[...truncated 349.07 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e0de97cc-38dc-43e5-b246-c6a91ed1dbc6.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b1eb8f02-4c70-4d3c-a125-3d3b89c98692.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b54ca330-021c-438e-ae3e-74b47068e079.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/89cd8446-48d7-45f8-9e26-617355594367.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/01524f57-5bbd-444b-859f-f06fc94092e9.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a79f9041-5e29-4744-81ac-ab48d65deb62.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe1c6cd2-b074-4343-9dfb-b0a23bd87ee5.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1dcedb2-1a86-4bfe-9b6f-9f107053b7bd.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1852daad-4d28-406e-a303-2a4981bc9e0a.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/af8cc3e5-805d-42e2-85bc-15d0bdce81ab.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c803098-00e7-497c-aa43-7caac57da72e.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a5e7adb9-1eee-42f1-a88c-e6863b06dc7f.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2836bd2b-9034-40ba-9aac-de17d8bb5422.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a8237b29-dd2a-4de3-91d4-f645e9a6eae8.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35cebfc5-6709-4552-91be-20fe564badef.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a4916a86-d9ac-4771-ab39-d229b2faa575.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/377ddb8d-ba74-4b70-ade6-6c45c296699d.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e74bb906-f403-4ca1-9a6d-4886fa5b6375.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ab4ea06a-51c0-4afd-9a16-20a81deff7d3.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd7650dd-be11-4fbf-8d9d-d88b405e6034.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c5343c6c-2611-45e3-8eb1-2b8fba748a65.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d4570843-395c-40b6-ba0c-e92d9b16db5a.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0187fcc5-4d5f-403c-806d-71a9dfb2f598.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edce727e-58ff-43ae-96a7-bc5d891ef7e4.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31897e19-6dda-4a49-8c22-f093351443e8.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61c072a0-3e9a-48de-b226-f44a04335afa.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1aa4af9e-270e-4bd3-80c6-a36cef8507fb.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11216399-5019-452d-b9d6-614859fe99c5.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f3655f9-64df-4d3c-8bbd-1ea3f9884bbb.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b2e02fb-424e-4215-bdaf-d2cc4e5e6166.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4d578cc1-1b96-4328-b43b-8d1f4e7ab7a3.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f95fcbf4-cef3-4432-9be2-8433cf2cbd97.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7030a664-ac82-4273-8f3d-db28a0c90933.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/348e5fb3-2aa7-4f5d-982d-f21d2322403b.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d7e9ffd7-903b-4605-b93f-dba3f1eeb741.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b23f1896-da44-4359-9ca0-e3d6a0a3ca61.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d9fc979a-76ee-4bb2-a92d-a36ce4b97fe8.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/56b72e24-0936-4afe-bb88-d79d06c3c4a4.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/948706b7-b3b5-4058-b659-fe5b2ea8420d.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/926b88fa-3690-40ef-84db-1fa890cfb06d.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c36986a0-77d2-4784-8173-34469eba0ea6.jar
May 21, 2020 12:56:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/60db72ce-f93c-44c7-bbea-90fa878b524b.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2e5ce44d-8a8d-4ea9-9a10-b007030d0e71.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/67f24c8d-32d0-4c6c-ba5a-9ef41b00e504.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0abe166b-ce43-4996-8bb9-31331d39efd9.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f3c62c52-9ccc-49b8-a4fa-0ecca4313b2d.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d7eb4a9f-15a8-419c-a242-a60d3ade2206.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f48a9a11-57df-4feb-8f6c-e854a1ce902b.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ca1d2fbb-fd08-43ec-89f1-9b8cfbb72b81.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4a7320f4-d5d6-4cfd-8488-56a4a5ecdb6c.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5dce9934-e6c4-450b-bacb-4e88e18b0db4.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/92a90c1f-f693-4495-8a72-2e4af625a2a7.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/220db665-6fdb-4098-b6b5-6d46eec5ebd8.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c0cb0dc-cad3-45df-870a-20e746440a00.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/14efe48e-018a-4f5c-b175-dde1e5843e6a.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/eaefc3dd-775a-4dd8-81a5-4e721b8b917e.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a6b290e0-2825-4879-9607-a30433d1c141.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c28879f6-0abc-4a65-9a2e-cd65126520b0.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee77ffb7-3037-40dd-9624-c9d2bc13796a.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61c0e7ca-a7da-4b59-a0ce-090cb8c3c4a1.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6fc15cea-3d8a-4655-a074-46c3e4903b9c.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e583542e-3fad-4459-8232-465d3d52e33c.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/896fc18f-b210-4b69-9233-8e76e4b493de.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/67efda37-27a2-4896-af04-30a8181d58f5.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a8af047-c26c-4269-b53c-576b50cb78f7.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a82e98e6-c906-4387-ad3d-5203f28a8b72.jar
May 21, 2020 12:56:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/916cc0ce-27eb-4974-81db-acca66808037.jar
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 21, 2020 12:56:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 21, 2020 12:56:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84200 bytes, hash 3c906b125fadf69b210fab8602b3d5e5f34fe31a7a6bd5366c53c3c966b5e75a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4341c842-1885-44c9-860a-84cc1160e595.pb
May 21, 2020 12:56:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 21, 2020 12:56:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-20_17_56_57-32905280744584623?project=apache-beam-testing
May 21, 2020 12:56:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-20_17_56_57-32905280744584623
May 21, 2020 12:56:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-20_17_56_57-32905280744584623
May 21, 2020 12:56:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T00:56:57.644Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 21, 2020 12:57:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:01.109Z: Checking permissions granted to controller Service Account.
May 21, 2020 12:57:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.132Z: Worker configuration: n1-standard-1 in us-central1-a.
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.755Z: Expanding CoGroupByKey operations into optimizable parts.
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.801Z: Expanding GroupByKey operations into optimizable parts.
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.842Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.954Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:05.993Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:06.030Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:06.075Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:06.534Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 12:57:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:06.627Z: Starting 5 workers in us-central1-a...
May 21, 2020 12:57:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-21T00:57:18.590Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 21, 2020 12:57:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:34.681Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 21, 2020 12:57:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:34.722Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 21, 2020 12:57:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:40.097Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 21, 2020 12:57:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:52.527Z: Workers have started successfully.
May 21, 2020 12:57:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:57:52.563Z: Workers have started successfully.
May 21, 2020 12:58:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:58:36.578Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 21, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:58:36.799Z: Cleaning up.
May 21, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T00:58:36.898Z: Stopping worker pool...
May 21, 2020 1:00:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T01:00:37.422Z: Autoscaling: Resized worker pool from 5 to 0.
May 21, 2020 1:00:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-21T01:00:37.464Z: Worker pool stopped.
May 21, 2020 1:00:56 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-20_17_56_57-32905280744584623 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 45650204-3ba7-4d12-b242-68c7104aa82e and timestamp: 2020-05-21T01:00:56.050000000Z:
Metric: Value:
fields_read 4375276.0
read_time 25.594
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 21, 2020 1:00:56 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 17.127 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 31s
104 actionable tasks: 71 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/zytzwt4j7k666
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #527
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/527/display/redirect?page=changes>
Changes:
[github] Merge pull request #11596 from [BEAM-9856] Optimization/hl7v2 io list
------------------------------------------
[...truncated 349.50 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/85639fa1-b0ff-4b59-afad-eb3859df1725.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/43017e30-36b3-430e-bf38-6a162044762e.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e7b721c2-698d-420b-8967-7477f8e3c815.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d098876a-7907-4236-bb6b-b140d53f8e4c.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/19466d8e-2073-40a2-8fae-7f11c677d70d.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6540ee8e-c921-459b-808f-de4e29dfb09d.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/56eca306-5c4e-4eb2-b864-f23cef216dd4.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a9c5cd2-90a2-4bc1-8960-4c31eb2b594a.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/245c404b-9320-4c72-90bf-e5a556655708.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3037a573-85ea-42c3-b4c6-0c2e0cd04dbc.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/44236252-dbe0-4f03-87ce-2125df9b91fc.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e232168-52ed-493a-8a1f-39e914c045ea.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16313a19-f7ff-4378-ba9d-9f6973f9c863.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04d6dcc9-d243-4594-b8e3-bddd3030455d.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/494cb942-bb35-464c-8522-994ca6a11ce0.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3944c1e3-e632-453d-9d93-c3c505ce9c3d.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/febfc8df-d604-4538-9dbf-1485c8308cc4.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/08908488-22b8-460c-a622-947dcf5a2d4c.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/88087e97-9e52-48a8-9a6b-f21da7f4469f.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d524a172-ce05-4b8f-a6fc-00cc46fc703a.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/64c140d0-8b75-4f0d-a446-343f8d0d6b9f.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ba4178c-363d-4765-8d40-2f133c8edf0d.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c714bf66-7dcc-48d8-8f62-6c81ff79393b.jar
May 20, 2020 7:47:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/791801a8-605a-44dc-a954-6bbc6463c328.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/39120775-a347-4564-a109-53393012b2de.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b4b6446a-c6fa-4c81-9015-2f70dc10a63e.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0622788-093e-4573-91ad-a3ac88cdbbfb.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0c9fb8fd-b165-4b67-aac1-126602e1b8a9.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9af2241-d9e1-4b37-9e00-ef1f44999992.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3556656e-b096-4ff5-9097-7ee42945b32f.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e170f1c-9df8-4ff3-99e4-43bdb72fbcd7.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b174b032-7e56-466a-9962-fb5f0ac5214e.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/503d7e53-cb9b-40b9-aeef-6b856d462ea9.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa954ebb-6641-46f6-b74f-3699649584b1.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d131c19c-1a3a-4762-8224-ea74b5f1e420.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/273d3c27-bbd4-4524-83f9-884ed41a573d.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d131a85-cfbd-4012-8767-f865147ac5f3.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a5c79b6-ceb9-496f-8c3d-0940fb0f379f.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b2a76f3-7573-4a7a-832f-595d72e902f2.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/02130269-1f10-47d7-b7f1-3b63d9ceebd5.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/949a0c25-ac27-460e-a1a0-043a04cd2584.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9c5832fe-0d01-46bc-90a5-ed4007a8e710.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f37e210e-2b75-4329-a0d0-4d858025b98c.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a38d14dd-ef73-42c1-8ba2-106e852619e3.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fd5717b4-d7f1-4161-be4a-af650e3f28b3.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/093f82af-2f84-4c3f-b061-49846d57be64.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6983a8f0-5ede-4bc7-8d8c-a6c70080c7ce.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f33a0a84-581a-46e3-881f-4fa99472fc6e.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/db37851a-0324-4abc-b169-8dcb00ab8def.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/869f5646-f08f-4aa5-8ac9-ebb1a4d7a529.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ebf5ce22-85fb-4d57-90df-d19322e38e76.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/24eceba5-5a18-4e8d-8037-0ca763d9799f.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fe2c66ef-f8db-44ee-b275-c11f793d72e6.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c3cd2c58-f685-4ee9-a4f3-990649051ec7.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1e9b882e-b787-4dfd-ae02-addfa4eb1a3f.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/012e90cf-1b0d-480c-a1d6-367737782181.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/044ae3a0-6d66-42e3-9cc1-8b304bd4265a.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/502ae2f6-b00e-4dcc-9760-62497e0762b7.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9bbb3a82-d478-4e45-b2dd-fa5a1ea5b4a3.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/696b6e05-2c58-4a5e-821d-ea71d6e04412.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6e533cd-ff01-4db9-9ddb-6955423ec883.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a0e3883e-35ef-4471-916c-f4a5adeeb044.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef249b14-3538-4321-8ba6-ef8eccbc8bc0.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/49a3ac3d-f4e7-4f02-822f-07a5a5a1c4e0.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fa817207-a7ce-4a14-9d49-3c2408544c00.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5f8ca3b3-45ab-4ed1-bd04-87a6b3ba18ee.jar
May 20, 2020 7:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/809fee92-9069-4aff-9cae-7f72511101bb.jar
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 6 seconds
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 20, 2020 7:48:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 20, 2020 7:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84200 bytes, hash f6501e1587994d864298bbf0f755b892e3e98630f7d076e4c4873f15bed48def> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fd10e0c4-3f94-4214-8850-8ab7a8e0e609.pb
May 20, 2020 7:48:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 20, 2020 7:48:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-20_12_48_01-16655987502370675249?project=apache-beam-testing
May 20, 2020 7:48:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-20_12_48_01-16655987502370675249
May 20, 2020 7:48:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-20_12_48_01-16655987502370675249
May 20, 2020 7:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T19:48:01.930Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 20, 2020 7:48:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:05.132Z: Checking permissions granted to controller Service Account.
May 20, 2020 7:48:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:09.787Z: Worker configuration: n1-standard-1 in us-central1-a.
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.459Z: Expanding CoGroupByKey operations into optimizable parts.
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.489Z: Expanding GroupByKey operations into optimizable parts.
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.519Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.586Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.620Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.643Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.670Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:10.950Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 7:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:11.022Z: Starting 5 workers in us-central1-a...
May 20, 2020 7:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T19:48:21.005Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 20, 2020 7:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:39.586Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 20, 2020 7:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:39.614Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 20, 2020 7:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:44.942Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 20, 2020 7:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:59.526Z: Workers have started successfully.
May 20, 2020 7:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:48:59.554Z: Workers have started successfully.
May 20, 2020 7:49:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:49:33.510Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 7:49:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:49:33.698Z: Cleaning up.
May 20, 2020 7:49:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:49:33.781Z: Stopping worker pool...
May 20, 2020 7:51:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:51:35.814Z: Autoscaling: Resized worker pool from 5 to 0.
May 20, 2020 7:51:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T19:51:35.849Z: Worker pool stopped.
May 20, 2020 7:51:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-20_12_48_01-16655987502370675249 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): fc00adc5-1b1d-48e0-8dbe-3763e08b9ec5 and timestamp: 2020-05-20T19:51:41.127000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.793
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 20, 2020 7:51:41 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 58.951 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 10s
103 actionable tasks: 71 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/sy27n3cnk2noo
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #526
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/526/display/redirect?page=changes>
Changes:
[piotr.szuberski] [BEAM-9633] Add namespace parameter to MetricReader
[piotr.szuberski] [BEAM-9633] Add more parameters to pubsub matcher
[piotr.szuberski] [BEAM-9633] Add pubsubio performance tests
[piotr.szuberski] [BEAM-9633] Add jenkins job for python pubsubio tests
------------------------------------------
[...truncated 348.36 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ef1f3cf-55ca-4c2f-b7bd-978051aa6496.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3f57e364-f1c2-424a-9079-8bc8366aea23.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7fb6ee7d-35df-4c1a-b2fe-a82b4d286f78.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a5bd83c-4c05-459b-af7f-b57415125b87.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6240e8c-3516-4484-adb9-aec52273a1e3.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/925f273e-7e27-4e15-b4af-a2f020edc1b6.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c1265980-7d0b-4acb-9647-ca6b45711e05.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d020008-4380-422d-83c0-b7079f5627e4.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/02f16472-a64e-40fe-8361-c37340067fab.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f571dd52-47e7-4e37-84a0-49a4c70343f3.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b7f3d276-ce9a-46b9-9d0e-3bcbe20c166e.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6cac82b4-8667-4f5d-a58d-f1c3f83ad59a.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/22fb15fa-7cad-4728-af12-6949169e62aa.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8f280b3c-b8f7-4dd1-b5e0-c526b70e4540.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d47087c3-464c-422a-92fa-9a3ac32d1ae8.jar
May 20, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/650e7957-02d4-4d93-a67a-5d098d7dd6d4.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/249e820b-4624-4dcb-8c3d-4a35cb05f27e.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5da67fb4-a5b9-4304-8f65-fafaae01bf32.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/07e13e57-27eb-4846-8a45-0b5718bc1bd5.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/961bc118-baca-46bd-b768-af3ac3ce9da7.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/97a66bc5-f6f3-4989-b18a-491f76bf868c.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/310415b4-3a2c-4f9f-bc5e-3c9f22710d16.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd3c5491-b035-4433-a27b-5c56edb3885a.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/16378851-65bc-4548-bfc9-3ba16766ef30.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/495833a9-b6c1-4bb4-9f66-5b06a15cbb10.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/919886ed-217b-4675-8597-1e9eeadca6df.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b759cd3b-b04b-45f1-8121-92eed529cb8f.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4e9b50a8-313d-4fd1-83f4-d52f526b7206.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/007276a9-0af9-4f99-804f-38a52893d292.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df94fd28-af7a-4213-af57-6d748f30089a.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d7090ed-d10f-47c2-a9f2-640c7791eb34.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54374f8c-a2a0-4610-ac4f-831a9250678a.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70bb0eaa-b1cd-4365-bd19-de6cb30ea920.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7278989d-5c32-4d1e-8950-c7a9d4ad9740.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6146db95-ec96-4034-ae01-342b303f8d6d.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/78872e10-254c-4fa7-baea-ae5842d523ae.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0509bb9-7ef7-4e18-99bf-d5d7353e9db2.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/56f73c2b-557f-4366-8fe0-df0fd45bde3c.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3cc5afd0-b0d9-4323-a7dd-fb51acb75d52.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ab64b8a8-b803-4f9b-a97e-9512025c4cc3.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9b36d09a-ca04-4b79-a1e8-44901ec17018.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/43b9bcfb-f365-413f-a838-65a379bd3dad.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ae237ff-c591-4712-84a0-cff22a5df91b.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0fca71c-d444-430e-b85e-8bfcc1b8f38a.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5bba407e-a389-423c-8021-235e6871724d.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8ae1fbff-2085-45f6-a335-04093d33bae9.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/004d4c36-3775-4720-b060-ba5f695e71b8.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c4323c86-4520-4900-be6a-dae232464a1d.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d70884a-b577-49f8-9ab7-3b93a0e65a99.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/247985b8-0530-49aa-bade-709ce190388c.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f578da2e-733b-476c-b160-3a47d534ea1d.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/db8f7439-7db6-4430-9232-a6c13dba5de3.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2a645507-1c27-45fe-b054-4a0320251eb3.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/133fe9b2-521c-4034-b684-34f3ba42b231.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1168d28d-42fd-4b77-84ac-0d72d9d221dd.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0821c842-4325-440a-9df3-379c2b71434e.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27acbe49-7681-4865-abed-145524b7aae3.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a973eddf-fc8e-4f95-a7b3-78f12485dcab.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/dae9b229-d4aa-45cb-bc7f-8b7c8e9680c7.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9c3c2fac-e057-4f45-8d2a-e3b4682682fe.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b5de7b52-5fb1-4edd-9706-66a7d3de5518.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0af9aea3-3878-477b-81b6-552bf897f321.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/117ad6fd-dd15-4b5e-8238-1f31254ca29b.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f2434002-0ef2-432e-9cd3-aeab18f93979.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1a5dbc74-acc5-4ab0-ac3c-87090f5606be.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/28a30b65-8c4d-4df8-bef8-8e01836edd6e.jar
May 20, 2020 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/92000f18-9aef-43d2-9c81-afcdb7e86ab9.jar
May 20, 2020 12:48:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54656ad7-0f8b-47c0-aca0-bb13819d6db7.jar
May 20, 2020 12:48:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5a858a5c-0a0e-4480-abf8-37407f9dcc8e.jar
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 20, 2020 12:48:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 20, 2020 12:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84202 bytes, hash 6444a3e44ad40cebdf9bd359dc550d0af134a628e0701ed0ab780468fd6a396b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3a48d936-43c6-44e7-99a2-ad8a26af3357.pb
May 20, 2020 12:48:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 20, 2020 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-20_05_48_02-8506016111015003398?project=apache-beam-testing
May 20, 2020 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-20_05_48_02-8506016111015003398
May 20, 2020 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-20_05_48_02-8506016111015003398
May 20, 2020 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T12:48:02.556Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 20, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:06.802Z: Checking permissions granted to controller Service Account.
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:11.139Z: Worker configuration: n1-standard-1 in us-central1-a.
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:11.764Z: Expanding CoGroupByKey operations into optimizable parts.
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:11.824Z: Expanding GroupByKey operations into optimizable parts.
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:11.864Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:11.962Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:12.007Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:12.040Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:12.079Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:12.570Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:12.658Z: Starting 5 workers in us-central1-a...
May 20, 2020 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T12:48:28.401Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 20, 2020 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:48:39.840Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 20, 2020 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:49:02.901Z: Workers have started successfully.
May 20, 2020 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:49:02.939Z: Workers have started successfully.
May 20, 2020 12:49:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:49:37.751Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 12:49:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:49:37.981Z: Cleaning up.
May 20, 2020 12:49:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:49:38.088Z: Stopping worker pool...
May 20, 2020 12:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:51:45.468Z: Autoscaling: Resized worker pool from 5 to 0.
May 20, 2020 12:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T12:51:45.525Z: Worker pool stopped.
May 20, 2020 12:51:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-20_05_48_02-8506016111015003398 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f600370f-1bf7-42d7-8d5a-9065213ba66c and timestamp: 2020-05-20T12:51:51.112000000Z:
Metric: Value:
read_time 16.158
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 20, 2020 12:51:51 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 4 mins 7.559 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 18s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/n3uhvbqenur5g
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #525
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/525/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9900] Fix polling behavior in UnboundedSourceWrapper
[mxm] [BEAM-9900] Add test case for polling idle readers
------------------------------------------
[...truncated 346.90 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6980afc7-d7cd-4c7b-9c0a-a18a234d4d00.jar
May 20, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d2c209d8-4317-4a27-a976-d5a4e4636175.jar
May 20, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/11a23372-bc62-4ab3-9060-028e9b5f5cc2.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1e9fea07-6802-462b-a718-06d4d467c9da.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8b8e09f-4d86-4566-a18c-8f87cc9b5915.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d3a232a-528a-41f8-934d-f5a0eac1c8b1.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04bfad6c-5bdc-49ec-8979-99b3c256bce1.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0f9caff8-45ef-431a-878d-c9d7b4ceae1b.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/92c88d19-ea40-4c70-b202-827cabe5bc55.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8319f681-874d-4dc0-b975-d11ce4437613.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6866fa8-a6ca-485f-bf5d-5f88af01baf4.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a1053c0-1f88-4315-ae8d-2ddda0913bb0.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d2b14be7-be7d-412b-8766-d7ae80ed176d.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2d7343b0-1765-45e3-8874-11b97b260eb8.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9f936c1-6ecf-4bab-bce2-3abac13b6396.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9e08633a-e2a8-49e4-9624-b08122cabab7.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/866f0487-7b37-4bdf-9e0f-437eec10f783.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d8e53594-54f1-4d26-bbae-41b0829dcaac.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b7eb2e7-405f-4ca3-ae8a-d98fa1007e00.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9eb9b378-2433-43a2-9a0e-eeda2158c40d.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4fb62605-561f-45b1-b039-29fd3796946a.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9cbf347e-7ace-42b5-97a7-9ee63f147bbe.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/41427cf8-5e29-495e-9ed9-eb183fae2a61.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d1dfe61b-ebfa-4b79-9002-10dd589f6422.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/09d874b8-88d8-4173-8144-312e40c0c157.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9de0775a-50b1-4552-bd59-cb1eac1946e4.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cd135e42-f458-413e-aea0-be6de9245c00.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a990d0cc-e865-4382-83eb-1b8295bcf500.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5418e3b8-87c7-4521-a4d5-af93979c620e.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/89fce0d2-a1b8-4242-9e07-9ca84f5c6e9a.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edf31145-2cac-4582-8023-e3bd09cef715.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/354055b2-2b72-41ac-8279-ca0f48225949.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27269e9d-eb96-42d2-93c2-530ee678f506.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/705ccdd5-6615-46a5-9d55-bcecfc306d84.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ebfa2fe2-234c-4f76-85a0-5f73c493660c.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee234a7b-b225-4c16-bdae-772463b069b5.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d35c55c-715b-4433-90f0-c1602cca79f0.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c6b126a9-2f39-4bd4-9e9c-618b47156074.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ea6c4535-fb04-497b-9748-960dbb1f16ac.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/12f6c01e-5d06-42ec-bdfc-3dc121a4315b.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/75b52ef3-030f-4499-acbc-4c9ecec2a6e5.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cb383036-4b39-4c8a-a50f-a2c74c725f52.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/52850e8a-4ca6-432a-9d8f-ad41fd3239c6.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1ccb3069-e696-44ab-aa92-e7c043cdb733.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7402a586-199c-4029-a7d2-c79edcdee5ff.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0b7bb719-098c-4cae-83f4-4010f3808561.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c9387932-d8d9-468d-ab33-8cd09d2663d6.jar
May 20, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/555e5841-d1ed-4aa4-8516-6373b44ada7f.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e78a6164-6395-4456-9fca-259196af9f18.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/31806269-d3cc-448a-9f23-dd222facb92a.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee2f6ee3-4a0d-4dc9-bcb2-f5aee7fc3bc5.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f1d66669-3ab6-4147-9165-9c9c0d29c0a0.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/35046429-0303-4eff-b296-d47308ef29f2.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1f81a9d-e220-4dbc-b349-ed1582012c48.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6d62a58a-3428-45b6-8035-e8b5213dfda5.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/64d2f307-d1e3-4ee8-9ed0-75e07442c8fd.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0536a8a-cefe-4037-8ba3-2bebc2f6b079.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4c620df0-250f-4b85-a73d-d2508978cd5a.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/daad2056-2346-4555-b4fd-1005be2f06f8.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b23a5d5b-bdde-4f6d-aad1-58deeda30a31.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e969a799-ee68-4b5c-a192-ccf939e7ef9d.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a90efe64-05c2-4e4a-a09f-742d6dde37fb.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/328dde9e-3364-4112-84b2-64bb52dcb354.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1bf0bef9-07d5-447b-9e1a-465bf9f37a66.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/93738082-559a-467c-a80d-32cd8390be25.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4602178e-ee00-4796-a7c9-1bf347cfb0f4.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c071cc98-cb5c-48c8-b481-30a2aa44c008.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1930098c-3068-48a7-8319-e3cbdc0a795e.jar
May 20, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/326898cd-76dd-465b-a20a-0341ff0849ba.jar
May 20, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 8 seconds
May 20, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84201 bytes, hash 3b1987559d1038ee5913df9d9a58ef63259168164be393d89c4daed90c7a05c8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7f68c0f9-efbe-4ef0-89d4-0b1aaa50c4c1.pb
May 20, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 20, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-19_23_45_42-6669959928822271538?project=apache-beam-testing
May 20, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-19_23_45_42-6669959928822271538
May 20, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-19_23_45_42-6669959928822271538
May 20, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T06:45:42.983Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 20, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:46.973Z: Checking permissions granted to controller Service Account.
May 20, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.005Z: Worker configuration: n1-standard-1 in us-central1-a.
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.673Z: Expanding CoGroupByKey operations into optimizable parts.
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.766Z: Expanding GroupByKey operations into optimizable parts.
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.797Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.888Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:52.928Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:53.019Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:53.100Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:53.461Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:45:53.531Z: Starting 5 workers in us-central1-a...
May 20, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T06:46:13.245Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 20, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:46:25.373Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 20, 2020 6:46:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:46:48.254Z: Workers have started successfully.
May 20, 2020 6:46:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:46:48.290Z: Workers have started successfully.
May 20, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:47:24.946Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:47:25.150Z: Cleaning up.
May 20, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:47:25.217Z: Stopping worker pool...
May 20, 2020 6:48:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:48:56.569Z: Autoscaling: Resized worker pool from 5 to 0.
May 20, 2020 6:48:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T06:48:56.608Z: Worker pool stopped.
May 20, 2020 6:49:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-19_23_45_42-6669959928822271538 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 6fdc80ac-5007-428a-bb87-e38c611172cb and timestamp: 2020-05-20T06:49:03.093000000Z:
Metric: Value:
read_time 17.775
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 20, 2020 6:49:03 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 40.532 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 44s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/h5qen64hg5wau
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #524
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/524/display/redirect?page=changes>
Changes:
[rarokni] [BEAM-9770] Add BigQueryIO deadletter pattern Decouple .java and .py
[chamikara] Prevent Dataflow from starting multiple containers for the same image
[nielm] Disable grouping by default when streaming.
[github] [BEAM-9958] Linkage Checker to use exclusion files as baseline (#11674)
[daniel.o.programmer] [BEAM-9951] Using the builder pattern for Go synthetic config frontend
[github] [BEAM-9984] Support BIT_OR aggregation function in Beam SQL (#11737)
[github] Katas - Convert task description from HTML to Markdown (#11736)
------------------------------------------
[...truncated 347.98 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5eb613d9-e2a6-42d2-b926-5a0f509d09a4.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b6ce93e6-7045-4801-94d2-dca3447d24c9.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ea87ea98-2ad6-4716-b66a-3f419702926f.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/bfd64afd-b90f-43e4-bdb2-87c1163845a3.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d3a099aa-9584-4419-b91f-b9b13a0a0347.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d0abd749-ddf3-443d-a8bc-c69ddc8f0b1e.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ce4d068c-3931-46f8-9e59-26a615e7be7f.jar
May 20, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70526518-d87d-4eaf-a104-13c2536c9d45.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/edf988b9-a4de-4dc0-b608-801f602146b8.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/27567a61-c9f4-43cd-8f5b-acafa43e20eb.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c50dac2d-7c89-44d2-bd8f-a6cba49ce879.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/516c114a-d0f8-46e2-beb9-178c06598bc8.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/206251d2-447a-4dbf-8a45-03d2719d85b7.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/50239eca-cdb2-4e82-98ac-6317b94bcc1d.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c43cc6f-5c62-4f1b-919f-13ed0474497a.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/23c11609-f1f4-4d9f-bbe0-aa9c20bb5890.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/731985f4-8948-40cb-82f7-4684e9f32689.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/109ae881-9b57-4fe4-b0d5-7683fcf102a4.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/0968909e-b2d4-4a9f-9f30-5b29184a9eed.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/084f92bf-df5a-4f4b-9bea-defa19311188.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4fd530e8-fea7-4d7a-ac87-a53fb5cdeb54.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5637dc75-be16-416d-a587-b1022399b81a.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/13291743-0583-4b03-b1af-5acf7ab5f4b4.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9920be31-9e5e-4fe8-896d-bdef20531708.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/276dc08f-adb4-4a64-80c6-f56f46ff139f.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b94633fd-82b2-47b2-94e1-b662bb6cc778.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6fb41238-e953-419c-8dbf-3d52f46c5477.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f89d4649-e1b5-4e13-bc8b-9c106b46ffa8.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6065672c-4350-45a1-ab4e-d826395ffb27.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/70859621-c7cd-4a14-89f9-fc8c20979f76.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/72ab581f-ab1b-4af8-80f9-e5f39a6191c9.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fc9f8056-cf21-4749-8314-6165f2d3576e.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b5348445-7918-4d7e-a7bc-f7dfad6bc0c4.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e442f725-1d64-4d35-9ea2-d96a12b15ade.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/86377842-b7aa-47d2-b2f8-eb646ae4e866.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/382a9137-6f24-4826-b33c-c16063b80fac.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/533dd87a-c2b6-4128-a247-bc2602021afa.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61df9ea5-710b-4953-b182-d2939abd4e25.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/406857bd-03c3-4669-ac5a-9dcc98e32846.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/462664a4-a965-40d3-a261-edd2513dbc0c.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6e3d85b1-a2de-49de-bd60-d0ff8c66a296.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/efb38157-c546-4be0-b322-3fd592e92fc4.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a53dcf56-e34e-47b9-bd54-82bb9e457fe1.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8d59bff7-b9cc-46d2-bab2-640d0dd3ea51.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e1ad796a-6817-4738-a868-db4011c00863.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f19f556c-4fd8-42a7-95c6-3c3839ebe89a.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a607ad3-d29a-4386-83fc-ffb8fe47dfc6.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fd9255a1-7aaa-466b-ab5f-c61183a1f817.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a3ac93a-5804-4ceb-9248-01d7491a1f6d.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/48281764-66c7-4222-aa03-a640e0b025ef.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/90ac053b-9f12-4d33-96e5-30ed1e343aac.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06ad1c4b-4f9d-4e42-90fb-80fc28fa6451.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7e6a0eb2-20e5-4f02-806d-4050f24f1f60.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/df8432d4-9652-4723-92a5-d977ce89a4ca.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9cd1633a-6312-4626-aee4-4afd61844a9f.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1119fb33-5078-47fe-916e-99e150ea7a0d.jar
May 20, 2020 12:58:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3191a74d-4866-4b15-81f7-22ab9daa48f7.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/40309efb-eb1c-4749-9298-e576eacab0f0.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a1efdb8d-acdb-4b8a-9231-26f50f4515a0.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7a47539a-642e-413e-bfa9-dc10bb9c905c.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/38c5065a-88ad-4a97-b5c5-94a7c87671aa.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/334bb2d8-d2ae-4d4c-b05b-980e2ba0feab.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b43024b8-cbf3-45e7-a90c-95011be8cb36.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4d33321d-9d68-4abd-8afa-c2955a56c34c.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6c227b5-efcb-41d5-87f1-49c9ba2554e2.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3cdaa5fc-d638-4a32-ad74-3d76cc399956.jar
May 20, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/642c3883-18ce-44fb-bfc6-67b1bbd03586.jar
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 5 seconds
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 20, 2020 12:58:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 20, 2020 12:58:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84200 bytes, hash 39687d9b8a2257dc23367883a3462d22d8fe5c710e4d7a569d789e3a393d3bb7> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-e756f8a7-b412-470b-80e6-80dd03d6b019.pb
May 20, 2020 12:58:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 20, 2020 12:58:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-19_17_58_33-5370053800264344712?project=apache-beam-testing
May 20, 2020 12:58:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-19_17_58_33-5370053800264344712
May 20, 2020 12:58:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-19_17_58_33-5370053800264344712
May 20, 2020 12:58:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T00:58:33.663Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 20, 2020 12:58:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:37.026Z: Checking permissions granted to controller Service Account.
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:41.950Z: Worker configuration: n1-standard-1 in us-central1-a.
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.656Z: Expanding CoGroupByKey operations into optimizable parts.
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.704Z: Expanding GroupByKey operations into optimizable parts.
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.734Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.800Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.827Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.869Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 20, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:42.898Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 20, 2020 12:58:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:43.312Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 12:58:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:58:43.400Z: Starting 5 workers in us-central1-a...
May 20, 2020 12:59:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-20T00:58:59.956Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 20, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:59:13.483Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 20, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:59:13.521Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 20, 2020 12:59:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:59:18.906Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 20, 2020 12:59:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:59:33.859Z: Workers have started successfully.
May 20, 2020 12:59:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T00:59:33.893Z: Workers have started successfully.
May 20, 2020 1:00:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T01:00:08.959Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 20, 2020 1:00:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T01:00:09.344Z: Cleaning up.
May 20, 2020 1:00:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T01:00:09.455Z: Stopping worker pool...
May 20, 2020 1:02:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T01:02:19.727Z: Autoscaling: Resized worker pool from 5 to 0.
May 20, 2020 1:02:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-20T01:02:19.772Z: Worker pool stopped.
May 20, 2020 1:02:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-19_17_58_33-5370053800264344712 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ccda8485-7ad1-4cf7-bf30-ad0b809d434d and timestamp: 2020-05-20T01:02:25.213000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.811
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 20, 2020 1:02:25 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 4 mins 9.757 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ozkk2ysyfrmyk
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #523
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/523/display/redirect?page=changes>
Changes:
[sruthi.sreekumar] Updated Flink runner version in NEXMark documentation
[kcweaver] Upgrade Dataflow Python container versions.
[github] [BEAM-9383] Staging Dataflow artifacts from environment (#11039)
[github] [BEAM-1589] Added @OnWindowExpiration annotation. (#11350)
------------------------------------------
[...truncated 350.60 KB...]
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.10.2/73d4322a6bda684f676a2b5fe918361c4e5c7cca/jackson-core-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a9bd2dd6-2154-4160-b24e-9b3374af8e54.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.10.2/3a13b6105946541b8d4181a0506355b5fae63260/jackson-annotations-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b149f9e1-9d92-4738-af81-d0001c90d13c.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.perfmark/perfmark-api/0.19.0/2bfc352777fa6e27ad1e11d11ea55651ba93236b/perfmark-api-0.19.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9f9ad139-7f4e-4edf-ad7b-2284b8bde62d.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-guava-26_0-jre/0.1/f309c3047ca99428e567afa42d233fb3e839bde1/beam-vendor-guava-26_0-jre-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/afa65fd4-f2a1-4427-99ab-8332789228a8.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.10.2/8a6a6ad573b48dc3b623414719428ecbfeb259a3/jackson-dataformat-yaml-2.10.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1acc89cd-3577-4449-92c9-b4befa111a87.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-bytebuddy-1_10_8/0.1/16661d7186f2d10440f1bb1c1e27e51f5306abe8/beam-vendor-bytebuddy-1_10_8-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ee45b193-cd3a-4174-b4ed-a6d4071049e7.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.xerial.snappy/snappy-java/1.1.4/d94ae6d7d27242eaa4b6c323f881edbb98e48da6/snappy-java-1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/945e93db-81bb-4f24-8e43-ab5a950a5891.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/83b90185-edab-4fdd-afd5-305edb43a528.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.8/c4f7d054303948eb6a4066194253886c8af07128/xz-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/06e0fb57-4739-4dd9-8206-cb20825d24d8.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-core-asl/1.9.13/3c304d70f42f832e0a86d45bd437f692129299a4/jackson-core-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ca9728c-871e-45b0-b522-eab6d9aa2c73.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-mapper-asl/1.9.13/1ee2f2bed0e5dd29d1cb155a166e6f8d50bbddb7/jackson-mapper-asl-1.9.13.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c7456fb7-a5c8-4499-9b7a-051e0ec59505.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a46bd4fe-cc85-42ea-b36a-b70dd84b456c.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.thoughtworks.paranamer/paranamer/2.7/3ed64c69e882a324a75e890024c32a28aff0ade8/paranamer-2.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8c3b43c9-4416-40d4-be1c-bd60a661d759.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/87dee749-f1fa-4681-9c35-e6a5cf2b292b.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/google-extensions/0.5/4f1d862216754651fc1e1f2b614746810e68a4ff/google-extensions-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/63fa6b7e-ec8f-4149-a313-ba03fc536065.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.20/b8df472b31e1f17c232d2ad78ceb1c84e00c641b/commons-compress-1.20.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c7e894a0-7e59-4012-b669-e8ef575341ef.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger-system-backend/0.5/b66d3bedb14da604828a8693bb24fd78e36b0e9e/flogger-system-backend-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c381e87e-d168-4411-a263-b258c7ccd3a2.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.7/5be124948ebdc7807df68207f35a0f23ce427f29/auto-value-annotations-1.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/013299bb-6eca-4dc8-b16b-cdb2dbfa7251.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.12/21ebaf6d532bc350ba95bd81938fa5f0e511c132/httpcore-4.4.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/f6747343-0e44-45d1-a39e-67c90a211961.jar
May 19, 2020 7:25:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d9d3f97a-616b-4c7c-a141-8a687e56aca1.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.1/a6a2b2df72fd13ec466216049b303f206bd66c5d/error_prone_annotations-2.3.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/417f747f-50ae-4743-b86b-57cb98a00469.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.8.6/9180733b7df8542621dc12e21e87557e8c99b8cb/gson-2.8.6.jar to gs://temp-storage-for-perf-tests/loadtests/staging/22e2e1e0-459a-4c0d-80e5-5675b4f083aa.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.android/annotations/4.1.1.4/a1678ba907bf92691d879fef34e1a187038f9259/annotations-4.1.1.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ca8d625-e552-4a91-bcab-5b954ae0e42e.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/2.0.0/518929596ee3249127502a8573b2e008e2d51ed3/checker-qual-2.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/188e89b8-fd7f-44c7-9427-44b9f5175e26.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.mojo/animal-sniffer-annotations/1.18/f7aa683ea79dc6681ee9fb95756c999acbb62f5d/animal-sniffer-annotations-1.18.jar to gs://temp-storage-for-perf-tests/loadtests/staging/aca4b13b-17fe-4eb8-8ac5-b9f407a6d019.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.42.Final/6e6fc9178d1f1401aa0d6b843341efb91720f2cd/netty-buffer-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/6a5ff10c-fb96-4c99-833f-1ee7a5fde737.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.42.Final/857502e863c02c829fdafea61c3fda6bda01d0af/netty-transport-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/58bc2f5c-ea90-43e4-9dea-1fb49f3e7907.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/javax.annotation/javax.annotation-api/1.3.2/934c04d3cfef185a8008e7bf34331b79730a9d43/javax.annotation-api-1.3.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ef1fb8e0-1e80-4a0d-905f-3bae4cf86112.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.kryo/kryo/2.21/9a4e69cff8d225729656f7e97e40893b23bffef/kryo-2.21.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9a85d149-fb0e-41db-8384-30b6ddc09047.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-socks/4.1.42.Final/dd355f01dc00f93aaebe805b05026d7cf57c60ec/netty-codec-socks-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/cae5caac-73e7-4483-a964-91c8afe49bdf.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.42.Final/b1d5ed85a558fbbadc2783f869fbd0adcd32b07b/netty-codec-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/827f09be-1971-40f8-acb4-fba93e001e94.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/42.2.2/7ebd60d15eec1f9e796d68212121d92e3dd566b2/postgresql-42.2.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/04e2256a-75f2-4686-bdb7-c2316ff21779.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/args4j/args4j/2.33/bd87a75374a6d6523de82fef51fc3cfe9baf9fc9/args4j-2.33.jar to gs://temp-storage-for-perf-tests/loadtests/staging/34d07d0c-4f4f-443e-afff-b62d2ad044a4.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.github.luben/zstd-jni/1.3.8-3/4438dbd483a242a169fd9b51c4820163bde710d0/zstd-jni-1.3.8-3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ac60d9f-e00a-4daa-873f-53e876fee826.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.65/be28c46df75712f1fece48639c05633c8217f71/classgraph-4.8.65.jar to gs://temp-storage-for-perf-tests/loadtests/staging/19c6695e-7763-4302-b0a8-130382757724.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest/2.1/9420ba32c29217b54eebd26ff7f9234d31c3fbb2/hamcrest-2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/98369ce8-651b-4fcc-b3bd-9bb4907ef0fc.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/79f2bee8-392c-4d97-be17-da498c956b11.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.2/d8b5e02b891e16d8be9e6d19bfeacc987168426d/powermock-core-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3c49555d-92fb-4ac1-a1e9-520ea498e6a5.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.2/609f2960a138e376df8700fa44e26cabfe960f45/powermock-api-support-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a7ec2173-5dac-4bca-9362-ffa8132b44ea.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.2/79df0e5792fba38278b90f9e22617f5684313017/powermock-reflect-2.0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/46b945cb-ab85-4080-9c68-9ac38c5e4515.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ognl/ognl/3.1.12/a7fa0db32f882cd3bb41ec6c489853b3bfb6aebc/ognl-3.1.12.jar to gs://temp-storage-for-perf-tests/loadtests/staging/595b1b36-fa8e-48df-9bb5-50debe26f54f.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1d32f024-b780-43b3-b42c-5dd5cce05484.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b90fc8ca-d1ae-4480-9612-23ad12a36d6d.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/4b3378a6-99f1-4ac4-b34b-4b98943d11a2.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b50d8c8a-df63-462a-a159-5ecc2c707ae7.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/deb6681a-bd60-4a21-a70e-3dafd267f943.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/ru.vyarus/generics-resolver/2.0.1/2182e67f161ddbe3ff8cb055bb54398354fda3f5/generics-resolver-2.0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/25986073-3054-49e0-ad5e-6a079beba487.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.6.2/bd1b74a5d170686362091c7cf596bbc3adf5c09b/log4j-api-2.6.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/906e73ed-6626-4102-8d1e-8db3ac60d91b.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.24/13a9c0d6776483c3876e3ff9384f9bb55b17001b/snakeyaml-1.24.jar to gs://temp-storage-for-perf-tests/loadtests/staging/61f406c4-2527-4a1d-a1c9-86d92e778416.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.conscrypt/conscrypt-openjdk-uber/2.2.1/59a346d64c0ddca750c5c877e274d5e6278e53ce/conscrypt-openjdk-uber-2.2.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/74aab082-f15e-4cf1-b4ff-27f5feb14691.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp/okhttp/2.5.0/4de2b4ed3445c37ec1720a7d214712e845a24636/okhttp-2.5.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c964efb5-2afa-48bb-b920-89971a3ac544.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.flogger/flogger/0.5/dd7449aa4aba72969ed409b96996d19906d534d7/flogger-0.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/7b44f4ed-57aa-4473-9375-7dcb90dc4cea.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-compat-qual/2.5.5/435dc33e3019c9f019e15f01aa111de9d6b2b79c/checker-compat-qual-2.5.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/b65a4843-32c3-4d6a-b98a-05552eb197a0.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/22c86c52-7adc-419f-bcda-f1b79e073fa5.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.42.Final/ccaacf418a9e486b65e82c47bed66439119c5fdb/netty-resolver-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/5c1e34f1-5a7c-4033-adc5-94af2004b0ae.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.42.Final/e02700b574d3a0e2100308f971f0753ac8700e7c/netty-common-4.1.42.Final.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ed752321-e6af-463d-b6d2-a04d4cb26270.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/e3ccb36e-faa1-457a-b1c4-7bbd722f8486.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1fd2ff10-fa97-4070-8f95-62b63bb554db.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.reflectasm/reflectasm/1.07/76f11c94a53ee975a0d9154b325c408b210155bd/reflectasm-1.07-shaded.jar to gs://temp-storage-for-perf-tests/loadtests/staging/faf40c10-d31a-4925-8feb-5abac0013ab3.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.esotericsoftware.minlog/minlog/1.2/59bfcd171d82f9981a5e242b9e840191f650e209/minlog-1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/c3b59817-2e43-45ea-b140-05c4e9d99a02.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/a97f2a07-2a99-408f-9f6d-df48682e45b8.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/1682bee8-a22c-4966-bdf1-2264bd3d987a.jar
May 19, 2020 7:25:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/8a83a9a3-c1b0-4072-b3b0-21f011755194.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/4.0/659add6efc75a4715d738e73f07505246edf4d66/asm-4.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/d7102751-2f93-46b1-9fc5-900d1c482d47.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.24.0-GA/d7466fc2e3af7c023e95c510f06448ad29b225b3/javassist-3.24.0-GA.jar to gs://temp-storage-for-perf-tests/loadtests/staging/54821de4-2b33-424b-aedb-b3711bbe571e.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/3ff40c19-6afa-488c-82c5-dc4a1fae9d96.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/9d48ef9a-afb6-46fa-b68a-10f4879104ab.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.9/122c7cee69b53ed4a7681c03d4ee4c0e2765da5/commons-lang3-3.9.jar to gs://temp-storage-for-perf-tests/loadtests/staging/2bb13620-64a1-477b-b633-28193d594d59.jar
May 19, 2020 7:25:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.javaruntype/javaruntype/1.3/26ba963f4b20c751e07b58b990bb41bf850622d8/javaruntype-1.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/553b7049-fb1f-4674-8a6e-2f14c283231b.jar
May 19, 2020 7:25:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 1 files cached, 209 files newly uploaded in 7 seconds
May 19, 2020 7:25:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 19, 2020 7:25:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 19, 2020 7:25:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 19, 2020 7:25:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 19, 2020 7:25:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 19, 2020 7:25:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <84202 bytes, hash 2edd916f90d4cb65d85a89e82c6bf6a301754e58735dcacd2d72a20615841e74> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-a7c42bf1-ec0e-49de-bb01-5e7c068719f2.pb
May 19, 2020 7:25:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 19, 2020 7:25:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-19_12_25_40-3211917342399236840?project=apache-beam-testing
May 19, 2020 7:25:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-19_12_25_40-3211917342399236840
May 19, 2020 7:25:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-19_12_25_40-3211917342399236840
May 19, 2020 7:25:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T19:25:40.895Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 19, 2020 7:25:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:44.221Z: Checking permissions granted to controller Service Account.
May 19, 2020 7:25:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:49.482Z: Worker configuration: n1-standard-1 in us-central1-a.
May 19, 2020 7:25:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.260Z: Expanding CoGroupByKey operations into optimizable parts.
May 19, 2020 7:25:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.306Z: Expanding GroupByKey operations into optimizable parts.
May 19, 2020 7:25:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.330Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.431Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.469Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.497Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.540Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:50.923Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 7:25:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:25:51.008Z: Starting 5 workers in us-central1-a...
May 19, 2020 7:26:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T19:26:14.003Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 19, 2020 7:26:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:26:22.295Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 19, 2020 7:26:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:26:37.808Z: Workers have started successfully.
May 19, 2020 7:26:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:26:37.841Z: Workers have started successfully.
May 19, 2020 7:27:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:27:15.862Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 7:27:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:27:16.092Z: Cleaning up.
May 19, 2020 7:27:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:27:16.182Z: Stopping worker pool...
May 19, 2020 7:29:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:29:28.279Z: Autoscaling: Resized worker pool from 5 to 0.
May 19, 2020 7:29:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T19:29:28.328Z: Worker pool stopped.
May 19, 2020 7:29:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-19_12_25_40-3211917342399236840 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): adbdc22b-0898-4016-9f36-79d075dc64a3 and timestamp: 2020-05-19T19:29:33.733000000Z:
Metric: Value:
read_time 19.996
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 7:29:34 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 12.376 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 16s
103 actionable tasks: 73 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/k4nvmwkgegawq
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #522
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/522/display/redirect?page=changes>
Changes:
[pawel.pasterz] [BEAM-2530] Sql precommit compile and run on java 11
------------------------------------------
[...truncated 293.49 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 19, 2020 12:53:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ls5RSQN9ruOR3qLYGq7pxQ.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-d2KYBIUgtnA-T8PlPYbBPA.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-96SmKpvgJ0Ag-bTsyCSg_Q.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-O57aFkwrT0DITTC4dEgxKA.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-2GibZu-9OkhL7KWkH7FxGg.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-wgbKhqA5zjvYy0ZfxC3g7w.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-6IEFBwo7t_w-cQLQdTyYpg.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-SpezkndMaAMiF5tcHYd19A.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-cSHrF6VqfB2rEHem_g6jHg.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-XpieT36_TFYiQ577mb85kQ.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-WBJ57hpmntRtAcOboFx3ig.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-wCIEZ_Zbll_gE0FQaQAEYQ.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-XBHMN0nbSsnTiwy4gVYekA.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-iDYEJnDCVgI9yHjaOJqCGA.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-R5rnXAgtB8b0cpWIHibrzw.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bG2fy1_IXmucBNS5Pwf5Ww.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-BWcOHwUFMG-TPNc9br8Jag.jar
May 19, 2020 12:53:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-k-MEVBS6Y5C2ueKnkzzU3w.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-7c3w-1WSaPaiRN6iQO-lNw.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-kGVOlUGrTcUDKv4ldoTXRg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ZhMPLyEIc4MRkyPaVsUGXg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-y_-3LQUs-7QHXL0_CuBDig.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-VGRfAIK8bPbBBK7hal2oGg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-LkLnqDzple3u3uVDYZbWaA.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Zgp1RDjVhV2_FZc5ldtaOg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9k5MX_7T1fhkmvzPUEfFLg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Pngb0mBTUL0hd5truFr0yg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-RnEoXkdK9FQKIHTj61x1sQ.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-vaQ9k7FGHyBS4nBKGGsQJA.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-asnVrvdhdRdgB15XLOWJeg.jar
May 19, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9k5MX_7T1fhkmvzPUEfFLg.jar
May 19, 2020 12:53:07 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9k5MX_7T1fhkmvzPUEfFLg.jar&uploadType=resumable&upload_id=AAANsUnta26Nw-8OfcASfSsZbXV-9plijrF6Q_9EUUcAuJ4A9kZadRX4TEjIH_hy-c8Pb--Spq5W-4oPnyE4y_JMx3VO4g44VQ.
May 19, 2020 12:53:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 19, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9k5MX_7T1fhkmvzPUEfFLg.jar
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 19, 2020 12:53:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12589 bytes, hash 875bu6MjC23ZBrB4RgkgTg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-875bu6MjC23ZBrB4RgkgTg.pb
May 19, 2020 12:53:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 19, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-19_05_53_14-18349367853216271097?project=apache-beam-testing
May 19, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-19_05_53_14-18349367853216271097
May 19, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-19_05_53_14-18349367853216271097
May 19, 2020 12:53:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T12:53:14.359Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 19, 2020 12:53:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:18.275Z: Checking permissions granted to controller Service Account.
May 19, 2020 12:53:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:22.594Z: Worker configuration: n1-standard-1 in us-central1-a.
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.233Z: Expanding CoGroupByKey operations into optimizable parts.
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.264Z: Expanding GroupByKey operations into optimizable parts.
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.296Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.387Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.422Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.447Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.484Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.810Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 12:53:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:23.883Z: Starting 5 workers in us-central1-a...
May 19, 2020 12:53:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T12:53:47.081Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 19, 2020 12:53:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:50.156Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 19, 2020 12:53:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:50.194Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 19, 2020 12:53:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:53:55.532Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 19, 2020 12:54:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:54:08.991Z: Workers have started successfully.
May 19, 2020 12:54:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:54:09.023Z: Workers have started successfully.
May 19, 2020 12:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:54:45.232Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 12:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:54:45.459Z: Cleaning up.
May 19, 2020 12:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:54:45.553Z: Stopping worker pool...
May 19, 2020 12:56:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:56:44.988Z: Autoscaling: Resized worker pool from 5 to 0.
May 19, 2020 12:56:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T12:56:45.028Z: Worker pool stopped.
May 19, 2020 12:56:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-19_05_53_14-18349367853216271097 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3652354b-93c7-43b7-a9ec-618a04216eda and timestamp: 2020-05-19T12:56:52.320000000Z:
Metric: Value:
read_time 17.665
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 12:56:52 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 56.362 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/imo2lanqk7oec
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #521
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/521/display/redirect?page=changes>
Changes:
[nielm] Populate all SpannerIO batching parameters in display data.
[nielm] Fix capitalization, clarify descriptions
[nielm] fix capitalization, clarify description Grouped
[nielm] Refactor to extract single method for popuplating displayData
[github] [BEAM-9339] Ensure that Dataflow's pipeline proto also contains the
[github] [BEAM-9363] TUMBLE as TVF (#10946)
------------------------------------------
[...truncated 293.33 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 6:46:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 6:46:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 19, 2020 6:46:44 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 6:46:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 19, 2020 6:46:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 19, 2020 6:46:44 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 19, 2020 6:46:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 19, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-k6lW7XyHmEWAjPsTTPxQGQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-Z89kyrokbPsGRq13P-hSRQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-WnP4Jp1j-l3XnFxroqWnLQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-oWdYcfxzKlFx1JAJ4uW5nw.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-t1yQU74OMMo8PFqi-l6lIA.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT--bWEBAmDMI82-R4c7orjRQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-NMmOq-9o6j1375pS_29lVg.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-caUBYs5_a8P7ScuReIYR5g.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Hcsa-qKWtIXDgf-xxlAaQg.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-OfQp4PY6MqxdbTqlMhyvcQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-dzxNpDfjagkTRogQxxY73Q.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-D56wtNmy6Zu9G5vbYVmx0A.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-SUkUmv5N5W77pXTnRLqJhg.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-UU2I7R17L8ZUcBi8HYv08Q.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-A3cSTNpl09kz8SOFtMp1RQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-opHLRONfNIVOg1ldnl1DWA.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ZqABw9A5NykRlovK0Fc88w.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-qIb4TL4_Sso8b2CwgJmyDQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-pxVwKmMsqnG4sQS-gPRQ5g.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Z1wa31wrVYNUiXE1OwVswg.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-G485X4qI6DOzKASVBSOYMQ.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-cPuK_GhxvdCawf4kXZ4iww.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-C316JSPxjJyHdbQfeb0nCw.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-S1rYLGxWVZ1Gas_SLuiGig.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-pz9bXvrHv28nXY4FV6Jz5A.jar
May 19, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-a7fljQqFh-BBefGX28i31g.jar
May 19, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-N6oYs_Omg5LJvaDoVqAWAA.jar
May 19, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-ICm2e2Oc9UIgeoI-78ErRg.jar
May 19, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iq8KiG_pzxRb26kl66G4pg.jar
May 19, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Ri7mZJKrxp07AqyTJ12Gbw.jar
May 19, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iq8KiG_pzxRb26kl66G4pg.jar
May 19, 2020 6:46:47 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iq8KiG_pzxRb26kl66G4pg.jar&uploadType=resumable&upload_id=AAANsUnxpv6NzQD9L3FY9XRZb6ytv8JbiDCvh_Jpft-8GGONxWPreqQlFSJOfgnYT_XOT_TsXQpTaD-xNMfm3wkXVRw.
May 19, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 19, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iq8KiG_pzxRb26kl66G4pg.jar
May 19, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 19, 2020 6:46:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12589 bytes, hash UuDlkOgv85vUwgKBk6lO1g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UuDlkOgv85vUwgKBk6lO1g.pb
May 19, 2020 6:46:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 19, 2020 6:46:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-18_23_46_55-15501155850917004108?project=apache-beam-testing
May 19, 2020 6:46:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-18_23_46_55-15501155850917004108
May 19, 2020 6:46:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-18_23_46_55-15501155850917004108
May 19, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T06:46:55.351Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 19, 2020 6:46:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:46:58.694Z: Checking permissions granted to controller Service Account.
May 19, 2020 6:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:03.832Z: Worker configuration: n1-standard-1 in us-central1-a.
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.443Z: Expanding CoGroupByKey operations into optimizable parts.
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.492Z: Expanding GroupByKey operations into optimizable parts.
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.520Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.614Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.646Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.686Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:04.715Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:05.060Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:05.132Z: Starting 5 workers in us-central1-a...
May 19, 2020 6:47:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T06:47:28.654Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 19, 2020 6:47:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:37.593Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 19, 2020 6:47:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:37.631Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 19, 2020 6:47:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:42.965Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 19, 2020 6:47:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:54.646Z: Workers have started successfully.
May 19, 2020 6:47:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:47:54.678Z: Workers have started successfully.
May 19, 2020 6:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:48:29.893Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 6:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:48:30.097Z: Cleaning up.
May 19, 2020 6:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:48:30.175Z: Stopping worker pool...
May 19, 2020 6:50:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:50:10.097Z: Autoscaling: Resized worker pool from 5 to 0.
May 19, 2020 6:50:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T06:50:10.129Z: Worker pool stopped.
May 19, 2020 6:50:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-18_23_46_55-15501155850917004108 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b382581b-359b-45c6-97b4-4356d96adb1a and timestamp: 2020-05-19T06:50:15.328000000Z:
Metric: Value:
read_time 18.728
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 6:50:15 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 39.52 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 37s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/23x7ozy77s4cq
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #520
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/520/display/redirect?page=changes>
Changes:
[robinyqiu] Support ZetaSQL DATE type as a Beam LogicalType
[github] [BEAM-9964] Update CHANGES.md (#11743)
[github] [BEAM-9577] Artifact v2 support for uber jars. (#11708)
------------------------------------------
[...truncated 297.71 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 19, 2020 1:05:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 19, 2020 1:05:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 19, 2020 1:05:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-b0Bcu12mo7AnyDaGwaMrgg.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-7A7DQ89lf51gt3uOqUXlww.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-kB43gDHScSL2i6ILbFKV_Q.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-_aRxiqw80Sr530rYi13q6w.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-uF0Gh5Mx7Pdh9C_Z30u_XA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Ztp4XSGivf6GGqXgFETRjg.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT--qwYmToupuW_gMoj_9Ksjw.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_Lj6UIynWchFd4hnK5tEmQ.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-7ziELA9DtRUyPrlQLIti7w.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-NLYPeSpmcOPRAqaeQwM3rA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-1_Ted1IRdU3WSfF17cxYXw.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-LYsmb7VM_t7A-77FIDKXjA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-3kOqMnWcjw-JbnDe_tXBCQ.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT--pJDXoJGuJCP0YhAaUax4A.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-a8JdyFZRbocaNEKgYcTrqg.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-jhRo5efwySwF5n99hsmjMA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-VWWy1w1_OENu0ExDwuS1JA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-V3kNM-jEvS0omtRvcbHShw.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-SdU1K6dhuML98Y8OCyC_IQ.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-_LvuprynfAsC-5XAAA04lA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-kiCTFvMwcQZJN3ORzfEgEg.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Cx7OLGpcFH8_JrTZT1TsfA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-HPcMLg_NpiZ20VMl-VUU1w.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-0HUb5p4FGI8Ybd4qK22CSA.jar
May 19, 2020 1:05:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-LNhd8_JT21sUSZe01J3cgQ.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Zi1u3L4r0hWtv05Qh9LTdQ.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-HGxYzSBRKFWijAPN-j2Pkw.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ej_9qzwUMVTFsaYYF3cmNg.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ej_9qzwUMVTFsaYYF3cmNg.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-cSDfiM-wC_jKMIaKelmc-g.jar
May 19, 2020 1:05:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ZpNaPrCWt1CsqxwvAeeibw.jar
May 19, 2020 1:05:24 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ej_9qzwUMVTFsaYYF3cmNg.jar&uploadType=resumable&upload_id=AAANsUmJ53mCa3Y7v1vBVxyh7JeRlPHlTWx1Q2xTQYgo1_fsLNbB4MXXK_rLFHCPbgCXeV9CzWDNAvn9qPLfci9LgH3YQWkI1A.
May 19, 2020 1:05:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 19, 2020 1:05:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ej_9qzwUMVTFsaYYF3cmNg.jar
May 19, 2020 1:05:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 10 seconds
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 19, 2020 1:05:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash WQ2hO2WIY2JYfnUbtWOW8g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WQ2hO2WIY2JYfnUbtWOW8g.pb
May 19, 2020 1:05:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 19, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-18_18_05_33-202800513227832911?project=apache-beam-testing
May 19, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-18_18_05_33-202800513227832911
May 19, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-18_18_05_33-202800513227832911
May 19, 2020 1:05:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T01:05:33.157Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 19, 2020 1:05:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:36.983Z: Checking permissions granted to controller Service Account.
May 19, 2020 1:05:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:41.689Z: Worker configuration: n1-standard-1 in us-central1-a.
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.352Z: Expanding CoGroupByKey operations into optimizable parts.
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.392Z: Expanding GroupByKey operations into optimizable parts.
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.425Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.523Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.558Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.593Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:42.624Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:43.233Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:05:43.322Z: Starting 5 workers in us-central1-a...
May 19, 2020 1:05:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-19T01:05:48.781Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 19, 2020 1:06:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:06.558Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 19, 2020 1:06:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:06.594Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 19, 2020 1:06:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:11.966Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 19, 2020 1:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:29.591Z: Workers have started successfully.
May 19, 2020 1:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:29.631Z: Workers have started successfully.
May 19, 2020 1:06:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:59.329Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 19, 2020 1:07:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:59.558Z: Cleaning up.
May 19, 2020 1:07:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:06:59.653Z: Stopping worker pool...
May 19, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:09:18.150Z: Autoscaling: Resized worker pool from 5 to 0.
May 19, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-19T01:09:18.202Z: Worker pool stopped.
May 19, 2020 1:09:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-18_18_05_33-202800513227832911 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 27e3eb78-a571-4ec9-b93e-7a0a1dced2f5 and timestamp: 2020-05-19T01:09:27.122000000Z:
Metric: Value:
fields_read 4375276.0
read_time 11.923
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 19, 2020 1:09:27 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.05 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 4 mins 17.659 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 4s
103 actionable tasks: 74 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/2roe3mizytyvc
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #519
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/519/display/redirect?page=changes>
Changes:
[mxm] [BEAM-6733] Add pipeline option to flush bundle data before
[hsuryawirawan] Remove all answer placeholder checks as they can be confusing at times
[hsuryawirawan] Update course in Stepik
[ihr] [BEAM-10018] Fix timestamps in windowing kata
[ihr] [BEAM-10018] Kata failing due to failed parsing
[ihr] Remove unused import
[ihr] Add missing dependency
[ihr] Fix member variable name in Kata documentation
[ihr] Fix placeholder location
[github] Python3 fix - convert dict.keys() to list before indexing (#11733)
[github] Updates google-apitools and httplib2 (#11726)
------------------------------------------
[...truncated 293.39 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 18, 2020 7:11:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 18, 2020 7:11:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 18, 2020 7:11:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-a2MdYfIh4SaGMIZszQu5-g.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-DbpBUx8RMdmKWrpdCG2P_g.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-coFX1jBKslyuu06DS7v9qQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-tkagjeUmuSTR5H8Mdk7bgg.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-3XarVlU1fl0oaiEi0SkE3g.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-GJb1qw43yYgADC_VSjLaxw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-GGbaw0dOHY0ZMzosL4LziA.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-dGpWagyJHReN0rsOEVDaHQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-mCTfaV7EwkfWbxEUnQmJaw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-xq3fONH2fJr2eUZZLVaMFQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-RThAY_VTYL2xNKEa3gsHjQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-ip5066gh5VfWeEhVzPI1ug.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-aFVjStOGKdjeuWzrkRUoqQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-SDNABVzDz_yf472RXH_s2A.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wqmW9cOP5ZUi6vZkESv0Xw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-fPCNyicLD1lWcgHtzgaDSg.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-EBLgK4tnwULHDJiQQNNXPw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-HyVUl1w95pjTjstYgmMcZg.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-VvoQWOW_Ei2CbZQttZT-2w.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-5RGkgUCvIFLkeV-BcanHSQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-b5TP8UleDo2kNUoorB1V9w.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Gch8tBMwojfTYjBpUYNQlw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-YvObsbLBCervYNVsmPD2hA.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-1GbPxTkwICWIC4FqkWOiuQ.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-Kjwa747aQBv1IiGkFHlcTw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vpEJeibXWaJDpz0xE63d5w.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-XoFI2fH2b8pMgOoHPs_sIw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-Nf7kQqYhCCzA9RaQELRn5g.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4mB0HeIgxovYEXCyHo0cbw.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-neYDaf9vnRbQskPsdOap_A.jar
May 18, 2020 7:11:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vpEJeibXWaJDpz0xE63d5w.jar
May 18, 2020 7:11:38 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vpEJeibXWaJDpz0xE63d5w.jar&uploadType=resumable&upload_id=AAANsUnvWYP2y5J8-AJBr1BwlEkhprqVlb_zVLrYg9d4bCdc2tIizmot1StUC7_V35JOpInmfIYZmOwF-wzj5jDL-vGoKQedbw.
May 18, 2020 7:11:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 18, 2020 7:11:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vpEJeibXWaJDpz0xE63d5w.jar
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 18, 2020 7:11:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 18, 2020 7:11:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash LrANO_P3owbp4LpaU-VUAg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LrANO_P3owbp4LpaU-VUAg.pb
May 18, 2020 7:11:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 18, 2020 7:11:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-18_12_11_46-3308804716482521747?project=apache-beam-testing
May 18, 2020 7:11:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-18_12_11_46-3308804716482521747
May 18, 2020 7:11:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-18_12_11_46-3308804716482521747
May 18, 2020 7:11:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T19:11:46.567Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 18, 2020 7:11:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:49.536Z: Checking permissions granted to controller Service Account.
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:53.314Z: Worker configuration: n1-standard-1 in us-central1-a.
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.049Z: Expanding CoGroupByKey operations into optimizable parts.
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.079Z: Expanding GroupByKey operations into optimizable parts.
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.104Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.175Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.214Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.254Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.292Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.615Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 7:11:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:11:54.696Z: Starting 5 workers in us-central1-a...
May 18, 2020 7:12:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T19:12:15.763Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2020 7:12:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:12:19.325Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 18, 2020 7:12:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:12:19.356Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 18, 2020 7:12:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:12:24.704Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 18, 2020 7:12:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:12:42.122Z: Workers have started successfully.
May 18, 2020 7:12:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:12:42.146Z: Workers have started successfully.
May 18, 2020 7:13:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:13:16.627Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 7:13:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:13:16.806Z: Cleaning up.
May 18, 2020 7:13:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:13:16.894Z: Stopping worker pool...
May 18, 2020 7:15:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:15:39.963Z: Autoscaling: Resized worker pool from 5 to 0.
May 18, 2020 7:15:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T19:15:40.035Z: Worker pool stopped.
May 18, 2020 7:15:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-18_12_11_46-3308804716482521747 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ea76c9a0-63c3-4598-a0fe-f415dbd93a68 and timestamp: 2020-05-18T19:15:45.850000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.913
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 7:15:46 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 4 mins 19.882 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 23s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/r73ocfmzpxzeo
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #518
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/518/display/redirect>
Changes:
------------------------------------------
[...truncated 293.34 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 12:50:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 12:50:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 18, 2020 12:50:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 12:50:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 18, 2020 12:50:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 18, 2020 12:50:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 18, 2020 12:50:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 18, 2020 12:50:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-D0c5W_orpWSUobuXm_pQfw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-7Le0krnWJFyhdrmSJOiK1w.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-etomzLOQ0JNwcdpXNjENSA.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-MgRkTfLaknWGxm5OKnqzOw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-b95Vw7IPJutxLlAvxCp4yA.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Iqk71qBo9u107Aqnbdyl1A.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-n03sLDP4GtUhC01nfHMxCA.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-3OJBDbsCMGl6gQTmm12-hQ.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-MXDF0xlZni9Io0oIA21cLQ.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-kjqyJj0u6ToVWXMKMbhO8A.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-00dmJ9z1SisSFnQJR5uAmw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-GTMzVqEo_Fg0DeM4L18gDg.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-fJyiYY5BAE6H0tqTi87Gwg.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-cIImdUVAa40etLjrZPmfiw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-VM103aff5hBi285VK5kyOg.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-hFtr2F6CQmc_P7Nq2UVenw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-BkEISEPiD80ClWrAjBKMXw.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-xe87WzNw5-spW1En_LTRhQ.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-KkhuozrqAAcQ-G2e3EsbGA.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-MC1T2jiMlX6qT8ayQAE3yA.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-WBrEzRFKJlebxLMYkiQQ-Q.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-uJeWUQxcrK9Ws6Tv3xNl8g.jar
May 18, 2020 12:50:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-vT-uQDCJ6c1IrKNPx7HnIg.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-3U5woKMWuarzdW5PxG1Sag.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-FZlcQf3egU3uoDdaOPnDhg.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-y27CMAmxMazqX5tvrwDoIg.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-k8BCKR-YRjD_SbOc6NH8Qg.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-y9vmO7bcBvLP8VyXvSF1zA.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-k8BCKR-YRjD_SbOc6NH8Qg.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-4NXLOJM6xC6p3WttGq_E9g.jar
May 18, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-a8SLK37rimNmTGc_Y_41UQ.jar
May 18, 2020 12:50:44 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-k8BCKR-YRjD_SbOc6NH8Qg.jar&uploadType=resumable&upload_id=AAANsUmIcvPc1S0X5YdV_4SrUlo_tB1sf1bgBnLRRe05b5HfBSlh1isQNDWypCa_ruOTuvvZb4He2TAVcgJAhPHgvPs0y5zyUQ.
May 18, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 18, 2020 12:50:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-k8BCKR-YRjD_SbOc6NH8Qg.jar
May 18, 2020 12:50:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 18, 2020 12:50:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12108 bytes, hash kIjO5a3gma2cjZAB2ONlBg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kIjO5a3gma2cjZAB2ONlBg.pb
May 18, 2020 12:50:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 18, 2020 12:50:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-18_05_50_50-3585792851532006754?project=apache-beam-testing
May 18, 2020 12:50:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-18_05_50_50-3585792851532006754
May 18, 2020 12:50:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-18_05_50_50-3585792851532006754
May 18, 2020 12:50:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T12:50:50.119Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 18, 2020 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:53.136Z: Checking permissions granted to controller Service Account.
May 18, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:56.478Z: Worker configuration: n1-standard-1 in us-central1-a.
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.073Z: Expanding CoGroupByKey operations into optimizable parts.
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.105Z: Expanding GroupByKey operations into optimizable parts.
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.137Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.226Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.263Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.301Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.338Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.676Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:50:57.750Z: Starting 5 workers in us-central1-a...
May 18, 2020 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T12:51:06.056Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:51:23.506Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 18, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:51:23.544Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 18, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:51:28.854Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 18, 2020 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:51:41.800Z: Workers have started successfully.
May 18, 2020 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:51:41.824Z: Workers have started successfully.
May 18, 2020 12:52:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:52:15.439Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 12:52:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:52:15.643Z: Cleaning up.
May 18, 2020 12:52:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:52:15.724Z: Stopping worker pool...
May 18, 2020 12:53:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:53:58.846Z: Autoscaling: Resized worker pool from 5 to 0.
May 18, 2020 12:53:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T12:53:58.881Z: Worker pool stopped.
May 18, 2020 12:54:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-18_05_50_50-3585792851532006754 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 5eb5ef6d-8f43-4ee6-af74-47e7ba8bec8e and timestamp: 2020-05-18T12:54:04.177000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.905
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 12:54:04 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 32.17 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/3ugjaaaxdo4to
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #517
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/517/display/redirect?page=changes>
Changes:
[github] [BEAM-2530] Implement Zeta SQL precommit compile tests and run on java
------------------------------------------
[...truncated 293.58 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 18, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 18, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 18, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 18, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 18, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-g3vrjwJzZG96FmSaRm-KQA.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-8KSJfYAksk_ptb2n61LXBw.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-A_BcAFG3wfWL6qGdq4EvVg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-KbSP34rYLYxdEVaTNelXTg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-5g_1eS-9yY4QkNuM_PFsNw.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-7xnS3Te9Jnd-mpPe2vONRg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-vdkYj3tgFVZrOL7bgRNAbg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-ncK-a4AKZ5SkOyJneoOHag.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-gF-V3D6PcT1FbxF5zfAeEg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-59mEHII-_pfae4ZLOYrPHQ.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-xYmFG9zbeBkWoJ4AnaETBg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-SxaSau_ImM33NRCequo_3A.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-WF4raMol0q6VreBHo6JqyA.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-s-T10Q1w5dbi-_v5HSSnKQ.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-AHackQlx75b1H5NIQvbQZw.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-rCuRxPQYO9AUR0JN27yOEg.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-My1CwpDkOalE0T154fEpxw.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-EKLPUSMNhFS2t3GnurKE0A.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-EBRbnGq6iA2pMyX94pO3Ig.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-dhr60SEMU0ctL993eEr3HA.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tMJbbwdqVIlb5h2F2m8a9g.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-lSt5prBGgTVK9Id2JDCtOA.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-lLwtcPnvugKTyhvLzwu92Q.jar
May 18, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ZFa3xAlhFh6M1sIZBLIl7Q.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4Q5-D-qUrb5sBhoETsBWSw.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-r0Vl6T7aW5oF29eufelw_g.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-jK7yjYB16N_R9radDAuhfg.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-c6Rthlsbzy2MeyuwrdO1ag.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-CYpRFmT6npnlcXbh2IVsnQ.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TnTKM4fjHvl-if3TyBMHYw.jar
May 18, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TnTKM4fjHvl-if3TyBMHYw.jar
May 18, 2020 6:45:35 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TnTKM4fjHvl-if3TyBMHYw.jar&uploadType=resumable&upload_id=AAANsUlsAj7o9pQDYkKg2gxBM_TRb2fDWhjKbYULMnSd4aX1AbrhdZtn9QOmuxP-RaYDXDYIN7i54h1r_BeZck_wQIs.
May 18, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 18, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TnTKM4fjHvl-if3TyBMHYw.jar
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 18, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash HurG9fgtxIePQALEbPSnIA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-HurG9fgtxIePQALEbPSnIA.pb
May 18, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 18, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-17_23_45_40-9133054765823072347?project=apache-beam-testing
May 18, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-17_23_45_40-9133054765823072347
May 18, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-17_23_45_40-9133054765823072347
May 18, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T06:45:40.771Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:44.410Z: Checking permissions granted to controller Service Account.
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.014Z: Worker configuration: n1-standard-1 in us-central1-a.
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.620Z: Expanding CoGroupByKey operations into optimizable parts.
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.656Z: Expanding GroupByKey operations into optimizable parts.
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.684Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.768Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.802Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.825Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:49.859Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:50.162Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:45:50.249Z: Starting 5 workers in us-central1-a...
May 18, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T06:45:59.665Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:46:18.585Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 18, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:46:39.061Z: Workers have started successfully.
May 18, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:46:39.087Z: Workers have started successfully.
May 18, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:47:10.948Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:47:11.140Z: Cleaning up.
May 18, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:47:11.215Z: Stopping worker pool...
May 18, 2020 6:48:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:48:53.703Z: Autoscaling: Resized worker pool from 5 to 0.
May 18, 2020 6:48:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T06:48:53.743Z: Worker pool stopped.
May 18, 2020 6:49:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-17_23_45_40-9133054765823072347 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3ac9beaf-5d77-4e12-855e-ec4fe2f02de4 and timestamp: 2020-05-18T06:49:00.237000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.255
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 6:49:00 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 36.944 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/45ui7huj73cmg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #516
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/516/display/redirect>
Changes:
------------------------------------------
[...truncated 293.55 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 18, 2020 12:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 18, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 18, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-a8T1ITF3U9x0hfQLVYq2bw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-mOv9TxVEN28kP4OG2Khvzw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-C8OWyQc10HHLgWixi4czRA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-237UYt91eo5xnuJQnat5Ow.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-KX1_dC9ldyfUSuqWGBOzYQ.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ZXs5-nF5NtC4ClXWvr46mw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-pkrzEcxaQ5aipCcb5BJc-Q.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-_i2ODIf5DG8V3ZNcRpFRyg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-pYeTuypWw5nJo1acsYd2cg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-vA_wNTvuo0NNxk1d3tOMRg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-iprxjejar_ERNjjOs3y-AQ.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-IkjQ90SaHpL5TfKS-IhLZA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-eeMQzqyEKFk4-n0OZvUsvg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-g7V24miX7XGj3TQM7fz9eQ.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-JJuDIthsR-CLatsTWAQEHg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-_4Zj0iPJ6GgQsgDDB3mjfQ.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-WsY58LQgXet5dpYOv9gmjQ.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-gzBdI6aacnUa7ytssVWDGg.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-WPxDouz-tBj2WGJqLhFOWA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-bvHTYu3jl7PRvVipsaqEfA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT--FX4ysytcdfct1tAZesE9w.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-aA0IKIDnXf3cvOhZ_9a-uA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-VgSENfdEAuBkz28uWkVlkw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-VXZvkzJZiuN84Bdgf-JcSA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ois5j7H1a1df_vmEztVgzw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-yLFqfBHQ1X1cdzwr7hvsFw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-g6t0HmicUHXd-ZTJJdLXIA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-r7YnaeGo2GVYeqTBjzRbMw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-oLt4OEyf_pgMG7PvBEi5wA.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-r7YnaeGo2GVYeqTBjzRbMw.jar
May 18, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ZmL0j2ii7A_WIVcipoXWaw.jar
May 18, 2020 12:45:42 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-r7YnaeGo2GVYeqTBjzRbMw.jar&uploadType=resumable&upload_id=AAANsUmhkOLqoBvsLIkNIiQEYRXZH6rdtN48Zj49i9Si87oxpvXtKAbRbj2Jwj9gAr9hRKfOfzmwWEWE1BHhSrHlVRZsTn2NTg.
May 18, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 18, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-r7YnaeGo2GVYeqTBjzRbMw.jar
May 18, 2020 12:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 18, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash ch38GvqNeWa5B03Al3iT1g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ch38GvqNeWa5B03Al3iT1g.pb
May 18, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 18, 2020 12:45:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-17_17_45_47-6775923474350046908?project=apache-beam-testing
May 18, 2020 12:45:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-17_17_45_47-6775923474350046908
May 18, 2020 12:45:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-17_17_45_47-6775923474350046908
May 18, 2020 12:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T00:45:47.960Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 18, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:45:56.701Z: Checking permissions granted to controller Service Account.
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.295Z: Worker configuration: n1-standard-1 in us-central1-a.
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.806Z: Expanding CoGroupByKey operations into optimizable parts.
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.848Z: Expanding GroupByKey operations into optimizable parts.
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.885Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.948Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:01.975Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:02.014Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:02.048Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:02.633Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:02.715Z: Starting 5 workers in us-central1-a...
May 18, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-18T00:46:15.173Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2020 12:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:27.813Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 18, 2020 12:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:46.899Z: Workers have started successfully.
May 18, 2020 12:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:46:46.931Z: Workers have started successfully.
May 18, 2020 12:47:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:47:17.282Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 18, 2020 12:47:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:47:17.471Z: Cleaning up.
May 18, 2020 12:47:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:47:17.553Z: Stopping worker pool...
May 18, 2020 12:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:49:12.052Z: Autoscaling: Resized worker pool from 5 to 0.
May 18, 2020 12:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-18T00:49:12.091Z: Worker pool stopped.
May 18, 2020 12:49:17 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-17_17_45_47-6775923474350046908 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 12e6c58f-9060-40a8-86a0-92084aa99855 and timestamp: 2020-05-18T00:49:17.722000000Z:
Metric: Value:
read_time 12.836
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 18, 2020 12:49:18 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 46.814 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 58s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/zussin7miteoa
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #515
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/515/display/redirect>
Changes:
------------------------------------------
[...truncated 294.65 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 6:45:39 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 6:45:39 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 17, 2020 6:45:39 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 6:45:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 17, 2020 6:45:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 17, 2020 6:45:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-uCj5x7RiKsALA3E78vOjjA.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-c_TAh_BKbtFPdBINfDIu5w.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-XZnMjpbMOSlwpBgmBSXKMw.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-QNQ3VOTFp-xrPCO8XcPICg.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-v3PQ1Fmu0UzrmHMJ1OIwYQ.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-miZDrQAcNGIozH_J0_RjAA.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-YMcbnisYKKp7C726HSVxZg.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Tf7pnPyDQTkgZrQ114-tdA.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-QmSAl9g-wXvSBzDwlt5P_A.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-pfzcM0VRQqsyQYLezN4tzg.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-xpvRgzaJKC8_hh9nHK0HWw.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-zeCDIlHuJo1w7kz8Qv8K_w.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-czPkTvULt5HJka6HT3jV7g.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-bkXisgRclob0DMIs0zTnUA.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-sPiF_FcPTCgTLX7YOCvefQ.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-a4ZpXosxKklEvVLJA1FSdw.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MO2SM_e_BnJqgCFHneAj2w.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-h1SXWlGdAUjbfqkIkSOy9w.jar
May 17, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-qLXK3zw0m-LVYrqZGTpdSw.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/joda-time/joda-time/2.10.5/7f1d89817cd20a32444d5ab4160f035ab9b864e7/joda-time-2.10.5.jar to gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.5-pkpUcYhGz4dDJMCWf3Tlfg.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-G1XpYBS8MUsKq5uA9JSVQQ.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-a8azyE8GR98UIbSS5-KpeQ.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-GejdF4tGG_WKlqb1vBWF8w.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-6ffYab6gpMw6Yecjk6B00g.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-bODro3jYBF5dl-j5SkXz1w.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-zatkniY9393nhbj5VwfWAQ.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-aao2b-fZkPIwdLI1X4dB-Q.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-r0xAbhQ2phoh_F2n0YhfSg.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-0cxza9Qxq69C6cnAIcFdCw.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lSNv2jlyjn8IyvfLQ3PBsQ.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lSNv2jlyjn8IyvfLQ3PBsQ.jar
May 17, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-yeDr5wUaSlSkLsFuk0ChBA.jar
May 17, 2020 6:45:45 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lSNv2jlyjn8IyvfLQ3PBsQ.jar&uploadType=resumable&upload_id=AAANsUkxP-b0ufg4ddOIX7OzhAkRidsDe1B-A7J9lXmRmozcQRq4lVIc2h8H-P78U7YNxtBcsJoV7NjWrwOhsYdbb7LlSIiosg.
May 17, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 17, 2020 6:45:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lSNv2jlyjn8IyvfLQ3PBsQ.jar
May 17, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 32 files newly uploaded in 8 seconds
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 17, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash XqXaefDsCzOSheflWfJbqQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XqXaefDsCzOSheflWfJbqQ.pb
May 17, 2020 6:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 17, 2020 6:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-17_11_45_52-5963503902207758080?project=apache-beam-testing
May 17, 2020 6:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-17_11_45_52-5963503902207758080
May 17, 2020 6:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-17_11_45_52-5963503902207758080
May 17, 2020 6:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T18:45:52.143Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 17, 2020 6:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:45:55.619Z: Checking permissions granted to controller Service Account.
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.021Z: Worker configuration: n1-standard-1 in us-central1-a.
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.580Z: Expanding CoGroupByKey operations into optimizable parts.
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.612Z: Expanding GroupByKey operations into optimizable parts.
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.636Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.694Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.720Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.747Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:00.774Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:01.045Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 6:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:01.105Z: Starting 5 workers in us-central1-a...
May 17, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:27.615Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 17, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:27.651Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 17, 2020 6:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:33.076Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 17, 2020 6:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T18:46:35.045Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 17, 2020 6:46:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:52.915Z: Workers have started successfully.
May 17, 2020 6:46:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:46:52.947Z: Workers have started successfully.
May 17, 2020 6:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:47:27.298Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 6:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:47:27.499Z: Cleaning up.
May 17, 2020 6:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:47:27.708Z: Stopping worker pool...
May 17, 2020 6:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:49:26.850Z: Autoscaling: Resized worker pool from 5 to 0.
May 17, 2020 6:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T18:49:26.890Z: Worker pool stopped.
May 17, 2020 6:49:31 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-17_11_45_52-5963503902207758080 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f41380e3-dc10-4ec0-9d13-eaec2bfcaf10 and timestamp: 2020-05-17T18:49:31.914000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.622
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 6:49:32 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 1.113 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 10s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ekvfzwisjclf4
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #514
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/514/display/redirect>
Changes:
------------------------------------------
[...truncated 296.15 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 17, 2020 12:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-REMZGROW2fNryWqEH5GuRg.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-c4xlKqa1UpNd8NZWJX2pOw.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-4GD3qMJuqjygFcyPdkeYTQ.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT--Gmh1ViG7YprBPMNdACo9Q.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-8xzOfkccTKRg7H16RWJepg.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-P65T2HeNbVnFyvvOgqHv6A.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-mhukLlku3FbaiKG9_luHQA.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-LBEl1HvA0a45TKmymwYqDw.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-jafCSBy9fMOfwEJqtMSlyA.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-eAauFwA22VPH0r__4I7UTA.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-IBpjN1JYWoSFLWmCkq7Gzw.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-El0IHFfbd14zkletcM0XPA.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-nNRcQh9K5GXS10xobfOxfg.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-dAx94tO6L2_RdyF2xCMfqg.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-WNXNwG1t6yuWWoW8AvgLDQ.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-csyD7hSLxze8Nwc4FskIug.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-vVtIhg4G17KisSZtn9RJAw.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-sJg_d2UCahzJddDykVqdvQ.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-yDxnBQUGKmhY-ZmUY9qLdA.jar
May 17, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-1OCBT8IM6me00YL7uHiyOg.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-VwimZRrS5hxTN1AjvOGevw.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-IFQRQ1KbsBvVssugpODKcA.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Vz4U8xuNAtd4HrhlnPwpmQ.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-yaycWWvkRKJKDnCVluNGeA.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-QUanwYEWJvAKlo9w6JBv_A.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-MUi82nopkB8aMXZi6u6bJg.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-XRrS1KdTTzmUK4JNSglhXw.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-desJyYGvRjybXoH-abr2WQ.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ql2h94sguj63RTtfjLSSaA.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Qfokr3t909p0ftEX_ohQJQ.jar
May 17, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Qfokr3t909p0ftEX_ohQJQ.jar
May 17, 2020 12:45:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Qfokr3t909p0ftEX_ohQJQ.jar&uploadType=resumable&upload_id=AAANsUkZWzWQ4hkfQkmchA9-EkFKVE5XX6uw8ZPfzfr-hfVNzs1HkdbHu6-zTUmMCASk47ZtUcDjVcrN0qWg8pJy_Bb6ensj7g.
May 17, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 17, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Qfokr3t909p0ftEX_ohQJQ.jar
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 17, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash xu9cJXbIYmrFMhkcVOrgaQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xu9cJXbIYmrFMhkcVOrgaQ.pb
May 17, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 17, 2020 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-17_05_45_39-11705326024656099642?project=apache-beam-testing
May 17, 2020 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-17_05_45_39-11705326024656099642
May 17, 2020 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-17_05_45_39-11705326024656099642
May 17, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T12:45:39.313Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 17, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:42.507Z: Checking permissions granted to controller Service Account.
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:46.433Z: Worker configuration: n1-standard-1 in us-central1-a.
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.022Z: Expanding CoGroupByKey operations into optimizable parts.
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.053Z: Expanding GroupByKey operations into optimizable parts.
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.093Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.160Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.199Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.234Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.270Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 17, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.698Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:45:47.781Z: Starting 5 workers in us-central1-a...
May 17, 2020 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T12:46:10.212Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 17, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:46:15.497Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 17, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:46:15.524Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 17, 2020 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:46:20.886Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 17, 2020 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:46:36.216Z: Workers have started successfully.
May 17, 2020 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:46:36.246Z: Workers have started successfully.
May 17, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:47:16.941Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:47:17.134Z: Cleaning up.
May 17, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:47:17.221Z: Stopping worker pool...
May 17, 2020 12:49:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:49:01.325Z: Autoscaling: Resized worker pool from 5 to 0.
May 17, 2020 12:49:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T12:49:01.364Z: Worker pool stopped.
May 17, 2020 12:49:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-17_05_45_39-11705326024656099642 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 659d3a78-b3cd-424e-9656-103c7191035d and timestamp: 2020-05-17T12:49:06.637000000Z:
Metric: Value:
fields_read 4375276.0
read_time 22.874
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 12:49:07 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 46.295 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
103 actionable tasks: 74 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/5od3a6ejha77g
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #513
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/513/display/redirect?page=changes>
Changes:
[github] [BEAM-2939] Ensure that we update the watermark even when no elements
------------------------------------------
[...truncated 293.44 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 17, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 17, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 17, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 17, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-iJ37qDGwuF-1wU4fZfnhyQ.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-SQGqkqQ47G1BsdikbB2eGg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-fMH8JDN1sVAff5dciL9hXw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-2Oc7Mx2RwQsaocZB5DjLqw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-diVamj6XFs15bJLoT24s4g.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-PBgq6znNYBtB_VPvUhl3jw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-c7Kao-8RxAAH5t8cGkqD7g.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-OrLYbPor8EjomOvTemHzpA.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-dPWoOrBi5rfMpAFQng_6Lg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-1j_9ysuP8KtWP7Ft2rk57A.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-owonGy9WdlXJi-gmuZsdMg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-wqo5HGPViDmzrHU04gO6BQ.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-9GR_qxTVbjX-DjZaSgmfwg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-e01mwpwtqHLvMmJWdkTWMw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-aQCLHFz5FRGAWO_c4KLv2Q.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-eCHdlrbw7h_8Bwbth-JeLw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-eE0l5Q55jEm3UaeyuCbTkQ.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-mNbxXJnkjGrZmOhJ3_yWZA.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-8zZ0L58MifNtXDmM8KYz1w.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-PWOh6gPOw5PWGDq8r3t7Hw.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Xdu2kvPZNXDTYag5BFa5Kg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-DZWPMv88RvZCaqN_yEhTcA.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-X3WGiLclEja4Qz7ag9GCqQ.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-D3HjqkYWJzISN-FirjWgPg.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-o84oGmWjlYqoCz6rh4UC5w.jar
May 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-G8FGGaQjQVoQRn6IPmdRtQ.jar
May 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VsKmYBWxuwHT6GsXwsYtdQ.jar
May 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VsKmYBWxuwHT6GsXwsYtdQ.jar
May 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-wcdRZzqa_0Ny-HEbSMOXUw.jar
May 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-AMP7QB2wRmLVPlODkiIq_A.jar
May 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-2gT9HRXTWL-JcBObVKl6Pg.jar
May 17, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VsKmYBWxuwHT6GsXwsYtdQ.jar&uploadType=resumable&upload_id=AAANsUnwVGak86EA5n8-NHHzmSUqyvyLAzoKqUj4oFUJa9jlJyHE-O7vgRX6RzpsAHRzzBCM1fiP_rXV9NGOWMnNwQ.
May 17, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 17, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VsKmYBWxuwHT6GsXwsYtdQ.jar
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash v1FYzFSLANekFzRwtu3COQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-v1FYzFSLANekFzRwtu3COQ.pb
May 17, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 17, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-16_23_45_37-13077120272890499696?project=apache-beam-testing
May 17, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-16_23_45_37-13077120272890499696
May 17, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-16_23_45_37-13077120272890499696
May 17, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T06:45:37.191Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:40.293Z: Checking permissions granted to controller Service Account.
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.070Z: Worker configuration: n1-standard-1 in us-central1-a.
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.683Z: Expanding CoGroupByKey operations into optimizable parts.
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.723Z: Expanding GroupByKey operations into optimizable parts.
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.762Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.844Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.879Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.913Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:44.945Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 17, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:45.318Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:45:45.399Z: Starting 5 workers in us-central1-a...
May 17, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T06:46:01.269Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 17, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:09.129Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 17, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:27.285Z: Workers have started successfully.
May 17, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:27.323Z: Workers have started successfully.
May 17, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:59.653Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:59.821Z: Cleaning up.
May 17, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:46:59.894Z: Stopping worker pool...
May 17, 2020 6:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:48:40.171Z: Autoscaling: Resized worker pool from 5 to 0.
May 17, 2020 6:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T06:48:40.219Z: Worker pool stopped.
May 17, 2020 6:48:46 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-16_23_45_37-13077120272890499696 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): eab29e66-d70e-4034-80a8-b3dcf846d4d3 and timestamp: 2020-05-17T06:48:46.212000000Z:
Metric: Value:
read_time 15.162
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 6:48:46 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 25.577 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 28s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/27ilgfsp7qg2q
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #512
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/512/display/redirect>
Changes:
------------------------------------------
[...truncated 293.27 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 17, 2020 12:45:27 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 17, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 17, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-7yYovyBLjr2bgI7vDVpocg.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-u7MBHM7KtRB4LEse5GKLEA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-n7lEhZjjGOIaLTo77zpkYQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-VOy3cb2qco3qgAK-ptmDcA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-sVt2sV_VMvxNzgGpQhtf9w.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-4MBSnuUP4JrW8Of3dwYAuQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-pCf7KsKA5R9M2KDtDl2ETQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-r7rAeWi816iaimIs9c4f0g.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-pqhW1F9BXFodvRZCR6oxbw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ONvjqChJsJd_GHiNYtClHA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-vkGksUjHWw2out7OR4aT3Q.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-eZrKFu5gYWv1Q6NlRjKxGQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-zpNfI_e68p6bmXhnj2BBtQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-l72NpSwJcFNVekLeK5h8mQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-keaYQ9JgIgcLFYyqM6Wn_g.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-x9oX4PwUDBTrNdGiCsVV3g.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-6ow-iDTQKS9hTkg-BXT0BQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-JkX7tXwu8VF-tV5kexzBSg.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-8IKkAOFb8M4d7ClveQHH8g.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-43HsKd9rhRZ6_jgJgFMUrw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-T2jUjpi9R8QnXWtwwsI9vQ.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-AJzSmdcrFj4DsFxxYBhXsA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-vLqNixhhQMdka_YC0VsPMw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-z3Ou8J2Th2Nz0jrlEzL-kw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-_qddOtEk2Qr3PQ1dvSC_0g.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-vCkO7MquUZmxOFiwHkNzLw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-dQ6UCch8SeQNIdBaQvC9jw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-J-9Qwj7EotwOMlu88uqSAw.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WPSc-VYImZ5BSLXihuciLA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WPSc-VYImZ5BSLXihuciLA.jar
May 17, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-vmqbqRyjww8FB8qVYWDyTA.jar
May 17, 2020 12:45:31 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WPSc-VYImZ5BSLXihuciLA.jar&uploadType=resumable&upload_id=AAANsUkldHHIgckaLwTBRHDN9kgqUE3oH1Ddc-caQ9BL-OiQZh943KIVlRghe89i5LBG5YR1W8KQHvcvzOgMIfzTJ4OBpNUzXA.
May 17, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 17, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WPSc-VYImZ5BSLXihuciLA.jar
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 17, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash vmKuWh-dtdZhCqleARjDKg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vmKuWh-dtdZhCqleARjDKg.pb
May 17, 2020 12:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 17, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-16_17_45_36-6034611536961666171?project=apache-beam-testing
May 17, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-16_17_45_36-6034611536961666171
May 17, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-16_17_45_36-6034611536961666171
May 17, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T00:45:36.367Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 17, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:39.434Z: Checking permissions granted to controller Service Account.
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:43.112Z: Worker configuration: n1-standard-1 in us-central1-a.
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:43.732Z: Expanding CoGroupByKey operations into optimizable parts.
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:43.835Z: Expanding GroupByKey operations into optimizable parts.
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:43.862Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:43.959Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:44.001Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:44.039Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 17, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:44.077Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 17, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:44.747Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:45:44.827Z: Starting 5 workers in us-central1-a...
May 17, 2020 12:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-17T00:46:05.322Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 17, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:46:11.161Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 17, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:46:11.193Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 17, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:46:16.532Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 17, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:46:29.161Z: Workers have started successfully.
May 17, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:46:29.195Z: Workers have started successfully.
May 17, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:47:06.284Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 17, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:47:06.510Z: Cleaning up.
May 17, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:47:06.619Z: Stopping worker pool...
May 17, 2020 12:49:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:49:03.315Z: Autoscaling: Resized worker pool from 5 to 0.
May 17, 2020 12:49:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-17T00:49:03.362Z: Worker pool stopped.
May 17, 2020 12:49:10 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-16_17_45_36-6034611536961666171 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): df969be8-aa1c-4f6e-9836-7678929fbd96 and timestamp: 2020-05-17T00:49:10.388000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.633
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 17, 2020 12:49:10 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 50.97 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/5iskcgl4hy2ik
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #511
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/511/display/redirect>
Changes:
------------------------------------------
[...truncated 296.61 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 6:47:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 6:47:58 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 16, 2020 6:47:58 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 6:47:58 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 16, 2020 6:47:58 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 16, 2020 6:47:58 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 16, 2020 6:48:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 16, 2020 6:48:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-wPn8nNdBnvPnfDz4u_aHlA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-NeR6twbk3N7mQzMH1V4qfQ.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-76cU7XzBzw1tz-r-u-hMwQ.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT--fnp-rEWQPP818krj2aTPA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-cjzNJoqP0sf94v2p-lZMHw.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-Pgb3ShKLFbSP8zTE8SAVqA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-J4IB9PxIKXJIikuKTQSNew.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-B6yc6ey92gn0mV6ziVb54A.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ewq8VRjSlknwFyXMml7TYA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-5gFUqfs_c6uRNoyNFyKe1g.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-IGvEJHIPZpLlSWTDS_bGEw.jar
May 16, 2020 6:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xHIs0FkyQaOVkF3l2YOpoA.jar
May 16, 2020 6:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-OJYn3gARTzdJe78GxFMhaQ.jar
May 16, 2020 6:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-vUkc3KZrC4LbFz9WLU6opg.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-dto1fYkNdbhpa-UvGzsYCg.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-iHMxh9vIuSAmPNk89EULvw.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-LK9Bjfaizb3TmXPXZfjwAg.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Wg_rh0_MozT6Anhy_D4VhA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-xo9zVMm6PzksDI17DB2YpA.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-l8xKOqM4ykc4_DeGqTOGmw.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-d3HFCCBxl9K1_uI4yGZeGg.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-zeTpT3ibljTQOp7iM9giFQ.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-EhoSB3je9i9l1vAU-ZQcEw.jar
May 16, 2020 6:48:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-_-yhy4_2xArCL6wwA6Dofg.jar
May 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-5gFUqfs_c6uRNoyNFyKe1g.jar
May 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Tp8mYbWacqE64CoT05rn5Q.jar
May 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-iC3AA5qvsKi7pyA3C_URDw.jar
May 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-2pJQLkxcGfL6uQ2Dbnjnzw.jar
May 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-nz-OgSk3VvextAPzOH6F9g.jar
May 16, 2020 6:48:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-9m1aDP7eOlI7-EAf9MRgLg.jar
May 16, 2020 6:48:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-B9XDfTNND1Du7jSRhyYgyg.jar
May 16, 2020 6:48:06 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-5gFUqfs_c6uRNoyNFyKe1g.jar&uploadType=resumable&upload_id=AAANsUkEWt1pjdSuUo-c-mWonpyFTKykkHkCCCKftiBop4Zj90VYZJQe7WKBFSyL5L9Kb4zdOQBXMa9-ESbjip9Bt8ZA62bFkA.
May 16, 2020 6:48:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 16, 2020 6:48:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-5gFUqfs_c6uRNoyNFyKe1g.jar
May 16, 2020 6:48:10 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 10 seconds
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 16, 2020 6:48:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash Y9j2LDjv41Pfgh5H3BW9NA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Y9j2LDjv41Pfgh5H3BW9NA.pb
May 16, 2020 6:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 16, 2020 6:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-16_11_48_12-1480684955754125035?project=apache-beam-testing
May 16, 2020 6:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-16_11_48_12-1480684955754125035
May 16, 2020 6:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-16_11_48_12-1480684955754125035
May 16, 2020 6:48:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T18:48:12.825Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 16, 2020 6:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:15.870Z: Checking permissions granted to controller Service Account.
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:20.201Z: Worker configuration: n1-standard-1 in us-central1-a.
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:20.914Z: Expanding CoGroupByKey operations into optimizable parts.
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:20.954Z: Expanding GroupByKey operations into optimizable parts.
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:20.992Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.083Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.122Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.161Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.215Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.575Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:21.666Z: Starting 5 workers in us-central1-a...
May 16, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T18:48:33.813Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 16, 2020 6:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:53.321Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 16, 2020 6:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:53.366Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:48:58.735Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:49:15.522Z: Workers have started successfully.
May 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:49:15.558Z: Workers have started successfully.
May 16, 2020 6:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:49:47.781Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 6:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:49:48.008Z: Cleaning up.
May 16, 2020 6:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:49:48.103Z: Stopping worker pool...
May 16, 2020 6:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:51:40.232Z: Autoscaling: Resized worker pool from 5 to 0.
May 16, 2020 6:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T18:51:40.284Z: Worker pool stopped.
May 16, 2020 6:51:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-16_11_48_12-1480684955754125035 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 709eb6dc-a823-4462-920d-ed564fb92750 and timestamp: 2020-05-16T18:51:45.907000000Z:
Metric: Value:
read_time 15.624
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 6:51:46 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.057 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 7.814 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 10s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/b4pizgz7lrypg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #510
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/510/display/redirect>
Changes:
------------------------------------------
[...truncated 293.67 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 16, 2020 12:45:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-wMIel1FAIyRAgc9YWefeoQ.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-sKLU1acrSpZjCk-ip7SuIg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-LsIcKt72Ks4s-p81ZErnBQ.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-nbrr56olXLE6XQ8_hr_iWg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-es5KEE5pSlsYuGpu3sChvw.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-9mXY_73uWnmlQivmqy8y0g.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-beJrasWs7_e6fZOwi2RKzg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-FC7pWkCaZenqTU-QHHredg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-0i0napC6gXPKQArKmR78gw.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-bmYljQYFDOcl-f89a6zpZQ.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-zoI5s-eG097sZIVc-8bXPw.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ANHYlIpLFGzNNvPuHeO6oQ.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-82l9lGYOXqIoo80Yo9i2dw.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-RFSgwvhyh8VzoZuk9xO81w.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-JomAcAs3-Ye9SOE0SgixSw.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-r7dMQ8dqzUEOh9XorxTSXg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-s2Q1zwhdkJNTfNirdAzSZg.jar
May 16, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Jg7seZsY2uqMZSeHRai12w.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-0wUUsjvEsPWDorTnZMNkGw.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-eFVdQK5LvIMr5qAnOvf4rw.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-S-LQB_8vIHoQCwrUe_eNNQ.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-aUG6k1chTJiEdb64Y4hu4w.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-X_OGG8oTAUJozAT7KBPRVA.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-OIIRj5NTQM9nMVX0OWrBDA.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-b7SujKLrqabB6GpNI-tcrA.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-p0GNVEn1oYSQgmRz8ihPVw.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JUy__djc4ylP6DcSZV2Frw.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-xhZV-thjCEZBsXe4cUSRwA.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-xhZV-thjCEZBsXe4cUSRwA.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-FxXeFUEwycU9qppTqAyXmg.jar
May 16, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-iasENio_qMoLsXmgb7Yogw.jar
May 16, 2020 12:45:33 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-xhZV-thjCEZBsXe4cUSRwA.jar&uploadType=resumable&upload_id=AAANsUlEiK7GOmxpax-DLIcpTqOCl3aGLZy2_fhZXbG4ER33ifKfVXJKJhJnzkh4z4M5W1rx05sw4AGTgB-tNqKLhRqtzlAArA.
May 16, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 16, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-xhZV-thjCEZBsXe4cUSRwA.jar
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 16, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash nzOcoKpy9neXACOXnVzCCA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-nzOcoKpy9neXACOXnVzCCA.pb
May 16, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 16, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-16_05_45_38-11075481724869046596?project=apache-beam-testing
May 16, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-16_05_45_38-11075481724869046596
May 16, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-16_05_45_38-11075481724869046596
May 16, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T12:45:38.248Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 16, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:41.621Z: Checking permissions granted to controller Service Account.
May 16, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:44.985Z: Worker configuration: n1-standard-1 in us-central1-a.
May 16, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.660Z: Expanding CoGroupByKey operations into optimizable parts.
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.691Z: Expanding GroupByKey operations into optimizable parts.
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.726Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.815Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.853Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.888Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:45.927Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:46.304Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:45:46.384Z: Starting 5 workers in us-central1-a...
May 16, 2020 12:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T12:46:22.179Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 16, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:46:25.432Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 16, 2020 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:46:40.927Z: Workers have started successfully.
May 16, 2020 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:46:40.953Z: Workers have started successfully.
May 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:47:18.459Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:47:18.680Z: Cleaning up.
May 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:47:18.769Z: Stopping worker pool...
May 16, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:48:47.977Z: Autoscaling: Resized worker pool from 5 to 0.
May 16, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T12:48:48.034Z: Worker pool stopped.
May 16, 2020 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-16_05_45_38-11075481724869046596 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 16312c8c-6976-4a25-8fb7-8e0ea917fc02 and timestamp: 2020-05-16T12:48:55.320000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.498
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 12:48:55 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 34.314 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 36s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/jwvcr67ohtc4k
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #509
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/509/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-9951] Creating a synthetic step for the Go SDK.
[github] Merge pull request #11725: [BEAM-10015] Fix output timestamp on dataflow
------------------------------------------
[...truncated 297.41 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 16, 2020 6:46:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 16, 2020 6:46:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 16, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6Q6d4HMB7KK2JtlUWwqpwA.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Us5kZCwvSJbCfDsPVdf_Pw.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ffbe9AhOtbLb1GFXSprHWg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-BnkM54iYSqfbja9YcbcEVg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-bVpox56-2gPnEQIC4XE2cg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-deAoAn6YGt5gvyOWzmrEew.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-OL5wpISUEFh1uJgcSWtOow.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-PQ8h7koijKOfn1gLVKlUJA.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-bxUYJehdjFJt0oRCbi_khg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-em4TGpxQan_MXB5offwqMA.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT--qz0rKOA9HBJT66xlNeLgg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-BauQjVyfm2q5XPUjC2npig.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Au-eAxw9gzF6JLnvE2cC0w.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-_cl_W64kWyQ0fhCvcWX-Nw.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Ae1LvlYQ0ikcObMSNrTggg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-kxxff-glq3uS0Xqu58gZlg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-NZccrlr4gqgGKOIyieG41g.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-iSzRCwwEqK1sVpQfdxqKPA.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-0Hie7zUnCSLYgnbJkG0L8g.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-pn3dRQpiBl6uEzzYV0dAtg.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-VOw8-xhnBYyS4i0Ldh8qJw.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-FzuGdiXDTygbvRtVS1a7-A.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-i-KRfQO2a5l9ApsLQcKRRw.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-X-YTlF70hKHtmioz-4Gbiw.jar
May 16, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-I5pPiPmIO8dmW9NwiOP3ug.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-efZiuR7CMESHWdGn3Yqflg.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6hJd6P2sbPv-gMhkNbdOZg.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-kJ5UhgSpqqQk-dMBjg0eBQ.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-7_WSV9PdYhX_LkPntXzqQw.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6hJd6P2sbPv-gMhkNbdOZg.jar
May 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-q60vRMA7juP1SMUsEjzlcw.jar
May 16, 2020 6:46:22 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6hJd6P2sbPv-gMhkNbdOZg.jar&uploadType=resumable&upload_id=AAANsUn7UgtUiZlOpCzOJXUeX1SEGSOaYtOrQq3HCsZFi1o9acRQWdH2wB3R2bapo8Dxxiddl_4_10l6V05FxkQNQXI.
May 16, 2020 6:46:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 16, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6hJd6P2sbPv-gMhkNbdOZg.jar
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 10 seconds
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 16, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash mrwwnY7IkDe4kqW4SI6xfg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mrwwnY7IkDe4kqW4SI6xfg.pb
May 16, 2020 6:46:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 16, 2020 6:46:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-15_23_46_31-15484039524443571922?project=apache-beam-testing
May 16, 2020 6:46:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-15_23_46_31-15484039524443571922
May 16, 2020 6:46:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-15_23_46_31-15484039524443571922
May 16, 2020 6:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T06:46:31.293Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 16, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:34.198Z: Checking permissions granted to controller Service Account.
May 16, 2020 6:46:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:39.224Z: Worker configuration: n1-standard-1 in us-central1-a.
May 16, 2020 6:46:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:39.916Z: Expanding CoGroupByKey operations into optimizable parts.
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:39.956Z: Expanding GroupByKey operations into optimizable parts.
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.021Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.102Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.139Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.174Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.210Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.583Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:46:40.663Z: Starting 5 workers in us-central1-a...
May 16, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T06:47:12.054Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 16, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:47:13.223Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 16, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:47:13.252Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 16, 2020 6:47:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:47:18.657Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 16, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:47:29.816Z: Workers have started successfully.
May 16, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:47:29.850Z: Workers have started successfully.
May 16, 2020 6:48:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:48:11.864Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 6:48:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:48:12.092Z: Cleaning up.
May 16, 2020 6:48:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:48:12.191Z: Stopping worker pool...
May 16, 2020 6:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:49:55.625Z: Autoscaling: Resized worker pool from 5 to 0.
May 16, 2020 6:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T06:49:55.676Z: Worker pool stopped.
May 16, 2020 6:50:02 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-15_23_46_31-15484039524443571922 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a0b94a43-0cc5-495a-afb6-36025cbf58f1 and timestamp: 2020-05-16T06:50:02.431000000Z:
Metric: Value:
read_time 22.074
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 6:50:02 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 4 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 54.396 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 44s
103 actionable tasks: 77 executed, 26 from cache
Publishing build scan...
https://gradle.com/s/7j6y7bntyabp2
Stopped 3 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #508
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/508/display/redirect?page=changes>
Changes:
[rohde.samuel] Change TestStreamImpl to a producer/consumer pattern
[apilloud] [BEAM-9522] Reject FULL JOIN ON false
[omarismail] [BEAM-9964] Setting --workerCacheMB to make its way to the
[omarismail] [BEAM-9964] Added testMaxWeight method in WIndmillStateCacheTest as per
[jiangxuehua1213] [BEAM-9136] Update doc about docker image license pulling
[bntnam.app] Modified the codeblock background
[github] [BEAM-1866] Enable validation of PAssert counts in Java validates runner
[daniel.o.programmer] [BEAM-9951] Creating a synthetic source for the Go SDK.
------------------------------------------
[...truncated 296.31 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 16, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 16, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 16, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-VsdErupBxsH8Wm66domMDA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-uPnXZTS2GvwVkrXLzv0SXA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test--0Ylf-QgdYrwDp-SM0ESoA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-7abcolpwy9108_QT6NTXXw.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-VyYq729kM8Nm42pklgy1GQ.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-un2mG0flalbqpxX852Emfg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-q_VGZtKuF1UYnNOyoDVodA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-JU00EGZ86_iYHrBxM0sUng.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-tVSrEf3CL1KHwDy3xAi52w.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-TNTrLPyyb8vbNvwOVp8nJg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-bUe45PwNQiWicbZ-GDPMCg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Xqk6rk3SMGojU3vgkI_h8Q.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-htTVvIM-c9Vet1DeB_4Xlg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-xKZCNT70LSyi2y4coZUQRQ.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-I2vlmI_3KCjBo04rWppFBg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tD4WkFlU4gwi17vMlLzDrg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-aSFEOIvZ6IOTPjQYJ_IxUA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-q2xofi544jZgJTZlOJoC5Q.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-n24r532qLuw29Nb8Hcqk7g.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-rJXt726p3LH2Drf1JvboQA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-XLQGFAZP2x7KDDA0-BkEnA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-qj-KRMZhtDfGGLhQ2qYLhw.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-obULqV_ARU0NLQ81tLXSvA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-Bf3SHCCl5IeQ7BPiGil4Ag.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-kGXwAys8nb0oT1J7XyWbPw.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-FfBS0s-LgaXGiF23qO5QnA.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QjqJsjHguoq796cEnxKkMQ.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QjqJsjHguoq796cEnxKkMQ.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-guIY35PHovYepLHrcjs0cg.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-l9Rxw2qauxt-2fybjHPT5A.jar
May 16, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-XoDnhxQwbrcTNuUEEao6og.jar
May 16, 2020 12:45:37 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QjqJsjHguoq796cEnxKkMQ.jar&uploadType=resumable&upload_id=AAANsUmtSHxmfnK-aSLyiUgVEPSzK8OA8Gccwg_ylDkZyNZ5mMjNPwuGuVD6fy1kS0nuItdgbA_YULT4IYao_JfADPN5iUBo9g.
May 16, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 16, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QjqJsjHguoq796cEnxKkMQ.jar
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 16, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash dHaB3P0qNTqov2AVoyDihw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-dHaB3P0qNTqov2AVoyDihw.pb
May 16, 2020 12:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 16, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-15_17_45_43-3790534522177940627?project=apache-beam-testing
May 16, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-15_17_45_43-3790534522177940627
May 16, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-15_17_45_43-3790534522177940627
May 16, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T00:45:43.238Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 16, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:47.007Z: Checking permissions granted to controller Service Account.
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:51.609Z: Worker configuration: n1-standard-1 in us-central1-f.
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.239Z: Expanding CoGroupByKey operations into optimizable parts.
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.288Z: Expanding GroupByKey operations into optimizable parts.
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.318Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.403Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.435Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.467Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 16, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.503Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 16, 2020 12:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.886Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 12:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:45:52.971Z: Starting 5 workers in us-central1-f...
May 16, 2020 12:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-16T00:45:58.078Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 16, 2020 12:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:46:17.513Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 16, 2020 12:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:46:17.543Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 16, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:46:22.984Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 16, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:46:36.588Z: Workers have started successfully.
May 16, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:46:36.630Z: Workers have started successfully.
May 16, 2020 12:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:47:09.846Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 16, 2020 12:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:47:10.024Z: Cleaning up.
May 16, 2020 12:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:47:10.104Z: Stopping worker pool...
May 16, 2020 12:48:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:48:55.901Z: Autoscaling: Resized worker pool from 5 to 0.
May 16, 2020 12:48:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-16T00:48:55.953Z: Worker pool stopped.
May 16, 2020 12:49:01 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-15_17_45_43-3790534522177940627 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 11c7a8b9-3027-4c99-bb12-f1220530b835 and timestamp: 2020-05-16T00:49:01.897000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.097
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 16, 2020 12:49:02 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 35.911 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 42s
103 actionable tasks: 73 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/gg47ngbmly7bo
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #507
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/507/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9966] Use dataproc staging bucket for checkpoints directory
[mxm] [BEAM-9966] Remove unnecessary Reshuffles
[kamil.wasilewski] Make Post-Commits status dashboard compatible with the latest Grafana
[mxm] [BEAM-9966] Propagate the keys to the load generator correctly
[github] [BEAM-2939] Ensure that we don't checkpoint the initial restriction as
[github] support isRelease tag for docker build command & update release guide
------------------------------------------
[...truncated 293.35 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 15, 2020 7:13:46 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-G-6xzzD40fvL9IiShvQqtg.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-i4CxxQjEeatNuD7wBZipAA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Mm-rs-V975N-HVTwrZUdMA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-VGOjuZQljQMOZ1RwDH3LBA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-43_HqR_6Hrs7dWrJTpKvHA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-wV1in6nVCaUh01jz_rxWtQ.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-FW-PGC4VArZfFeFJe1laew.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-D_Qd4t7NUrX2A5W5FCoUCQ.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-RVK7uUwrWDW2yB8Sd_eEvA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7sEMthhXd928S1FNf7fX-A.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-AtPce6ezL2FTqVMfG9y59w.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-faFHqLDPhYNp1w2NWAJALQ.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-LuaNYVVrHKmhNehut1J-AA.jar
May 15, 2020 7:13:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-NQkrvlcJ_AgDGUJhqvbXcw.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-Xwg2fWCoYUXaqSvM9wkEOQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Ylf51AChPmLC_bJaMCYtzQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-sBHHQF2btzHvDLiiXObtmg.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-VNtEBOUl3E4RLn6MxPuyHQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-7OyjiJ-uN7kXc1tpOc_TDA.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-fbpkO5k62KBt2ASCPrIQzQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-vEOij49i9sQMPQQ-7t2Ilw.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Cfz5vlYQUjTcqxMkbWxYGA.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ESEtaLSUF0-dBqCia55q_Q.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-4tETUt60HJ2ggFzlE8x0wA.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-IQxcy6QbeXbnZClYs1vuTQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Qo87_rp8JllCaFPhSgk5Qw.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-SVPn7xQphqlaP4IKOCviMw.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GsIiDBqrHJz8yhzum648ZQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GsIiDBqrHJz8yhzum648ZQ.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-CCgHvZ9v_TGFcXpCX_igEg.jar
May 15, 2020 7:13:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-LWW4nRsrxiFWHTlGLYXvMQ.jar
May 15, 2020 7:13:49 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GsIiDBqrHJz8yhzum648ZQ.jar&uploadType=resumable&upload_id=AAANsUkqZGCbVIdkasO-HsRLqVNYqa1rVnrJhRLnfl9SXIvpTb-9RlVvsEh6HOR5TtRQC1l8Uwu093RJmRdkxIUhUTs.
May 15, 2020 7:13:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 15, 2020 7:13:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GsIiDBqrHJz8yhzum648ZQ.jar
May 15, 2020 7:13:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 15, 2020 7:13:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12103 bytes, hash -8rnKK7ON1UnwpdkHUiuXg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--8rnKK7ON1UnwpdkHUiuXg.pb
May 15, 2020 7:13:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 15, 2020 7:13:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-15_12_13_56-9997444405417289339?project=apache-beam-testing
May 15, 2020 7:13:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-15_12_13_56-9997444405417289339
May 15, 2020 7:13:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-15_12_13_56-9997444405417289339
May 15, 2020 7:13:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T19:13:56.839Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 15, 2020 7:14:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:00.566Z: Checking permissions granted to controller Service Account.
May 15, 2020 7:14:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:04.675Z: Worker configuration: n1-standard-1 in us-central1-a.
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.417Z: Expanding CoGroupByKey operations into optimizable parts.
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.448Z: Expanding GroupByKey operations into optimizable parts.
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.475Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.549Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.582Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.617Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:05.654Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:06.126Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 7:14:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:06.202Z: Starting 5 workers in us-central1-a...
May 15, 2020 7:14:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T19:14:29.882Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 15, 2020 7:14:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:30.784Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 15, 2020 7:14:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:30.818Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 15, 2020 7:14:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:36.151Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 15, 2020 7:14:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:50.000Z: Workers have started successfully.
May 15, 2020 7:14:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:14:50.030Z: Workers have started successfully.
May 15, 2020 7:15:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:15:23.452Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 7:15:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:15:23.680Z: Cleaning up.
May 15, 2020 7:15:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:15:23.779Z: Stopping worker pool...
May 15, 2020 7:17:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:17:43.949Z: Autoscaling: Resized worker pool from 5 to 0.
May 15, 2020 7:17:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T19:17:44.006Z: Worker pool stopped.
May 15, 2020 7:17:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-15_12_13_56-9997444405417289339 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f8d5f4aa-cec7-493e-b2dd-d47009e47151 and timestamp: 2020-05-15T19:17:49.962000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.417
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 7:17:50 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4 mins 12.85 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 18s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/kpdybdvpobqge
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #506
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/506/display/redirect?page=changes>
Changes:
[pawel.pasterz] [BEAM-2530] PreCommit Dataflow examples - test compiled with java 11
------------------------------------------
[...truncated 296.43 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 15, 2020 12:49:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 15, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 15, 2020 12:49:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-j_TOufjDE49TXuJz5KxYyQ.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-i846QUjw2P1dx3VM7ekdwg.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-euKId_lRKYjWlaq4BIgr-g.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ve73V01UDVeEANXWrBGupw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-X-qgGZ_HowB336IrtWgPvQ.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-4BxWp_9tRKMsDIqBTAMGmw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Yup2YySyFnjewmNMMndDaw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-dJhe02rcqxdX-vv1QcZKFg.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-VeRjR_dnLgDi8_Vwue8dHQ.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-JbiJpziLMszGYyJJwBN89A.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-EzEn6YXcJ5J7Cgs6-hpeEg.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-dAYZELLOMgsn1g6kvtIWrQ.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-9x6Zxz4y3gr_KxPWOO5Z0A.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-vJpzTb2yLsJf5j7gZ7QpHw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-iNjKt8Iocv59ExGV3dUbPA.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-438iO9b8jC5zyWt54cw3ew.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-6r7k8OPESwFhdM5fdkgECw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Gipq2aflpNe_Ys00r9QDEw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-1oU5Rc_Bu9zLH0OY76P_vw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-2aNht4oMoILnPmFKs8uvDA.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-8-aTG2L7Rfx8uooxuNY8qg.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-H7MXWPbo9pLWaD9CXlUIhg.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-TOJ_AkcjjQDmOYWflmGmYA.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-78o1Vz3AUZ6JS3GmGfyTvw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-xfV42SWkssOhfGlyKkEimQ.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-cxjuz1lLTd7gKMLqfwbSEA.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--WSbwIsDDXX4uhE1swc07g.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-YMGHORjuQGXCOJIN-aNjsw.jar
May 15, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--WSbwIsDDXX4uhE1swc07g.jar
May 15, 2020 12:49:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-SgC38W6R9l0muy-D10UGZg.jar
May 15, 2020 12:49:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Bu7rx_QIm-QexI5hoPOB2w.jar
May 15, 2020 12:49:12 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--WSbwIsDDXX4uhE1swc07g.jar&uploadType=resumable&upload_id=AAANsUn1Sy8MrFdxNIDQACbcvQ8MmfO5h31TDC0vPHWdtBK_PCWNqtwJtjHqLetEF431zqK8PUjPRibWU_NPEGqK7Wo.
May 15, 2020 12:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 15, 2020 12:49:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--WSbwIsDDXX4uhE1swc07g.jar
May 15, 2020 12:49:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 15, 2020 12:49:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash X71IieRCqbAKDDqy4rZ13Q> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-X71IieRCqbAKDDqy4rZ13Q.pb
May 15, 2020 12:49:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 15, 2020 12:49:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-15_05_49_17-2373587938816976352?project=apache-beam-testing
May 15, 2020 12:49:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-15_05_49_17-2373587938816976352
May 15, 2020 12:49:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-15_05_49_17-2373587938816976352
May 15, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T12:49:17.896Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 15, 2020 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:21.814Z: Checking permissions granted to controller Service Account.
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:26.428Z: Worker configuration: n1-standard-1 in us-central1-a.
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.179Z: Expanding CoGroupByKey operations into optimizable parts.
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.236Z: Expanding GroupByKey operations into optimizable parts.
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.268Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.377Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.410Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.441Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 15, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.488Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 15, 2020 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:27.975Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:49:28.069Z: Starting 5 workers in us-central1-a...
May 15, 2020 12:49:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T12:49:55.945Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 15, 2020 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:50:11.343Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 15, 2020 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:50:11.378Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 15, 2020 12:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:50:16.754Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 15, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:50:30.789Z: Workers have started successfully.
May 15, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:50:30.827Z: Workers have started successfully.
May 15, 2020 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:51:06.462Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:51:06.696Z: Cleaning up.
May 15, 2020 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:51:06.819Z: Stopping worker pool...
May 15, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:52:40.063Z: Autoscaling: Resized worker pool from 5 to 0.
May 15, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T12:52:40.111Z: Worker pool stopped.
May 15, 2020 12:52:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-15_05_49_17-2373587938816976352 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ceee1919-7b0c-41d3-b0eb-4c622ecd5617 and timestamp: 2020-05-15T12:52:47.923000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.261
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 12:52:48 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 49.247 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 3s
103 actionable tasks: 73 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/uumtbsdv34fik
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #505
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/505/display/redirect?page=changes>
Changes:
[github] Update release notes link for 2.20.0
------------------------------------------
[...truncated 293.96 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tKf2R37cdXY-vWay4VfXmQ.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-CAwD6aGdug36Xvoy76t9dw.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-vV_Mw3mQyJr3A_JTjPtqUQ.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-JfDOmfdgAGJsmRF0_1m1-g.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-BRPs_6BE1oP7oei2YxT8Lg.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-n2itIIG1fmJCavV5r3Eb9Q.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-XU82a6zxAfIiwuNx4G82HQ.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-sjxt6SLoBGlYNhrcVjUJVw.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-4Xa-vN6_Wd-NRolAcgPPeA.jar
May 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-eJbR08GCU4fQReHt-rO1dA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-qFgkKMjiFQ2xG0qzgw1faA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-WXWn_j7_8fI9JIuZEn-HnQ.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Bsha9FvfmVe4B25ZIKhy3g.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ywVL7jtTbDbF8SjPq_6VgA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-rQB7vlaQpTDG6QuD3P7wmA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-VYXIf_j2CIof0BSNnq4xYA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-1kQhJtniPfTrfA5qOErPGA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-PZxp_yzCSHG0xgN0LRiIBw.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-bWw9_m99Ce0688tEGPAI6Q.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-qyU7flV7zcTAlfQRBZ0cuA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-iMzyOapZNFUeGgwj-Iz96Q.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-DYhB2FmVMoZpSJ0rLWJNTA.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-gyq-7eAROtFXN9wHy4qxCQ.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-22Ibz2zbsTCKDRGVw-8aJg.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT--nFT8cTuhDIV69FTZQKIzw.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-1lSQV2n2PzRQAH6tKEcdEg.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-eh0oEATm3ampdR98WeT7nQ.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gT7P3STR97gU8rS2nqp9Zg.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QoJet_A3tbeTltC7B03ICQ.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QoJet_A3tbeTltC7B03ICQ.jar
May 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-GBaPM6YiTUuITcX8kZREaQ.jar
May 15, 2020 6:45:34 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QoJet_A3tbeTltC7B03ICQ.jar&uploadType=resumable&upload_id=AAANsUlf-z1GzelKG2pVJaigjs-8NvWYW0n8L6v0M2mysL6xjjRX3-6KXurZQjAOoRjLczSPUqoaqOEYqeYh0W12a4o.
May 15, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 15, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QoJet_A3tbeTltC7B03ICQ.jar
May 15, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 15, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 1WZuUlzIlrI-QnpyjQidcw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1WZuUlzIlrI-QnpyjQidcw.pb
May 15, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-14_23_45_40-18212599015194558007?project=apache-beam-testing
May 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-14_23_45_40-18212599015194558007
May 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-14_23_45_40-18212599015194558007
May 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T06:45:40.284Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 15, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:43.565Z: Checking permissions granted to controller Service Account.
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:48.581Z: Worker configuration: n1-standard-1 in us-central1-f.
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.276Z: Expanding CoGroupByKey operations into optimizable parts.
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.313Z: Expanding GroupByKey operations into optimizable parts.
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.344Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.445Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.473Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.504Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 15, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.530Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 15, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.874Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:45:49.961Z: Starting 5 workers in us-central1-f...
May 15, 2020 6:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T06:46:00.945Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 15, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:46:18.682Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 15, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:46:41.402Z: Workers have started successfully.
May 15, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:46:41.491Z: Workers have started successfully.
May 15, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:47:14.620Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:47:14.838Z: Cleaning up.
May 15, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:47:14.925Z: Stopping worker pool...
May 15, 2020 6:49:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:48:59.982Z: Autoscaling: Resized worker pool from 5 to 0.
May 15, 2020 6:49:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T06:49:00.029Z: Worker pool stopped.
May 15, 2020 6:49:08 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-14_23_45_40-18212599015194558007 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0cfa6cc2-fef2-4759-8907-5fa6237a63fe and timestamp: 2020-05-15T06:49:08.362000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.593
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 6:49:08 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 45.913 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 50s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/vqmjf7dld3sda
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #504
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/504/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-9967] Adding support for BQ labels on Query/Export jobs. (Roll
[kcweaver] [BEAM-9993] Add option defaults for Flink Python tests.
[github] [BEAM-9941] Added a test of a GBK followed by a Flatten with an unknown
[github] Clarify pubsub IO comment about timestamps (#11672)
[iemejia] [BEAM-9833] Add yamllint config
[iemejia] [BEAM-9833] Fix .asf.yaml issues, sort labels and disable rebase button
[github] Clarifies an error message in Katas to explain what is actually wrong.
[github] Merge pull request #11210 from [BEAM-8949] SpannerIO integration tests
[github] [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable
[github] Update the range for pyarrow to qualify pyarrow 0.17.x (#11699)
[yifanzou] Run Inventory job twice a day at 6 and 18.
------------------------------------------
[...truncated 294.25 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 15, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 15, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 15, 2020 12:45:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 15, 2020 12:45:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 15, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 15, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-4rPsBwGlY5YoJWdHsHDJfA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-3hHTUGn8YP-7gyhc-IGYPA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-x7PFRl36zF11yn6XS1Q1yQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-IZfkn1GfbCQe3aHob6T0SA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-ovWcBWjnM5rGvv_EupmjFA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-WgAk1LOD0hgZ7evrwnSDvQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-13Jp2Tr2YDOBDQju7515iw.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-4eFEf7QHILczU5wbz0jZ9w.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-esWWNFqtcD3y9pOgmbF4kA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-wggGHwNtUe5x3byR5Kt2kQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-bWG4W5qHslpM4eCGPhFPvA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-J_IqPgmI4YD3EyH52Roiow.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-9RHd2TdxS9xBYtjzFn6q2A.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-1BaQj9NwunDXqkdWxNk_kQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Ad_7pr-7WDGW6kR9-11vNA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6OT99YbTpPY_CWRNzxCQPA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT--UTHkve8YiooKl7oCNEBdQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-TN20bildCoFsgvuGQ5xlNQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-E9M9fr13W08vQvVhT6Y6AA.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-A8Iz-Q8zjvd4c1jnCP2r5g.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-8zm0sp8n8I2-hkYCjPiRxw.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-AwLSzFWyaoriGFrzED9jPw.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-wLNGQXjOlICOKujVTDdmVg.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-3fOOGfiu47_zXuhNutKHMg.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-acJ0_P3KwIYIfUyjlUPVwQ.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-m3ydz_ln4Y25RnLCtU5e2g.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-hpE1-AieZORcRGC9D2S6Ag.jar
May 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fkMzTv2wKDTgYt3jyT5qSw.jar
May 15, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fkMzTv2wKDTgYt3jyT5qSw.jar
May 15, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-FemwfRXD43o-ZBWByBQH-Q.jar
May 15, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-1iRLmzy0tFBWjAP-Z_HGMA.jar
May 15, 2020 12:45:36 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fkMzTv2wKDTgYt3jyT5qSw.jar&uploadType=resumable&upload_id=AAANsUn1y1OnZ1mr0rHiTJ6MALfLYXVg9WGhHz41cyBhAThCoVfvq9QDkK6apHYSedRj9PYu6D_PNFruFRjNFonaNsg.
May 15, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 15, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fkMzTv2wKDTgYt3jyT5qSw.jar
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 15, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 15, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash GxEoBQwdjSxmaJ7et5mExw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-GxEoBQwdjSxmaJ7et5mExw.pb
May 15, 2020 12:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 15, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-14_17_45_43-2812440441695584739?project=apache-beam-testing
May 15, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-14_17_45_43-2812440441695584739
May 15, 2020 12:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-14_17_45_43-2812440441695584739
May 15, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T00:45:43.657Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 15, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:46.912Z: Checking permissions granted to controller Service Account.
May 15, 2020 12:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:51.520Z: Worker configuration: n1-standard-1 in us-central1-a.
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.249Z: Expanding CoGroupByKey operations into optimizable parts.
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.296Z: Expanding GroupByKey operations into optimizable parts.
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.328Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.421Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.460Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.488Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.522Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:52.941Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:45:53.014Z: Starting 5 workers in us-central1-a...
May 15, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-15T00:46:10.604Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 15, 2020 12:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:46:25.949Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 15, 2020 12:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:46:25.984Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 15, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:46:31.379Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 15, 2020 12:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:46:46.241Z: Workers have started successfully.
May 15, 2020 12:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:46:46.276Z: Workers have started successfully.
May 15, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:47:19.087Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 15, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:47:19.300Z: Cleaning up.
May 15, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:47:19.376Z: Stopping worker pool...
May 15, 2020 12:49:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:49:33.386Z: Autoscaling: Resized worker pool from 5 to 0.
May 15, 2020 12:49:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-15T00:49:33.429Z: Worker pool stopped.
May 15, 2020 12:49:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-14_17_45_43-2812440441695584739 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9780f300-bc8b-4798-b0f9-741b0e180c47 and timestamp: 2020-05-15T00:49:42.447000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.245
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 15, 2020 12:49:42 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 17.734 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 19s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/g7xdhygovezfo
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #503
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/503/display/redirect?page=changes>
Changes:
[github] [BEAM-9634] Add natural language analysis transform (#11611)
[github] [BEAM-9430] Fixes the bounds of initial watermark set to estimators
------------------------------------------
[...truncated 313.82 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 14, 2020 7:01:44 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 14, 2020 7:01:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 14, 2020 7:01:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-mAZQa89K2Hmodg6__Vd7Sg.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-VUxc1CVWmP7_XhSnQWoFyA.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ahVR2bvCWgz3Zw5N2Q2IrQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-p0bdbx9IHHU3PXE8dEsVaQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-fXj-IKJNUQ1CKnSdnARqkw.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-2uDsUNX4m0WYcATY9zEdlQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-xDQuHm2FHpokz9FG349i2Q.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-YMU5CHHwp54uQLr20wyIUQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-jy2KY06NSOq79MpylHabAg.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-h8I8yhhZJa8AlwGBSIcSgw.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT--Xn20m0RB6m4tkkPYRvJmA.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bE2yEsu8usjBcssQ6LymIw.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ZCHt1qqI6mTN_4uN0IPCtg.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-YSwmzM8fehkALGB6Op1FkQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Ipyp7xRwoauTixSq2Rs-IQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-YCFDf1dBQMBButmh5LfxkQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-92rZWcYcLqzgOZuaHrGplQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-2luyS4s23LFbVZfR5VTa5w.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-jqAJW7OXb0i3nRrxYOupyA.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-6VS1oCVFrZOMqyirFaSgIQ.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-201flnfyLROiP1jego5kIw.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-8dpypIVXtVGAgKt07DPVyw.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-vMvgCc3sPMrDYK9LcNVO1Q.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-GM8vHiw1S1E2j3p6jGpNBg.jar
May 14, 2020 7:01:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-onya30DbPdcYtqQePfqOlw.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-ne-E60LXmw6-De6rTlIMFw.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3GyIq34BJTBes9589mRUig.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TJZrLq4fXVlftXtvRYe7hA.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-_mPdkCBltUhms-AmnRsHjg.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TJZrLq4fXVlftXtvRYe7hA.jar
May 14, 2020 7:01:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-E8-nnJqbP2Wok33HNFSc5A.jar
May 14, 2020 7:01:48 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TJZrLq4fXVlftXtvRYe7hA.jar&uploadType=resumable&upload_id=AAANsUnZ0T_5l5NL0rI4JqCK9djNkeuJHXO92yTfY08JVmEex0TSrTFE4j_gf9oM_mkPKyaY-FUPWw2vdmuGiWI4NRQ.
May 14, 2020 7:01:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 14, 2020 7:01:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-TJZrLq4fXVlftXtvRYe7hA.jar
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 10 seconds
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 14, 2020 7:01:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash iW3KLU4ehhl9tIKDEnkCFA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-iW3KLU4ehhl9tIKDEnkCFA.pb
May 14, 2020 7:01:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 14, 2020 7:01:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-14_12_01_57-15504423999323501362?project=apache-beam-testing
May 14, 2020 7:01:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-14_12_01_57-15504423999323501362
May 14, 2020 7:01:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-14_12_01_57-15504423999323501362
May 14, 2020 7:02:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T19:01:57.785Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 14, 2020 7:02:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:01.090Z: Checking permissions granted to controller Service Account.
May 14, 2020 7:02:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:05.656Z: Worker configuration: n1-standard-1 in us-central1-a.
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.298Z: Expanding CoGroupByKey operations into optimizable parts.
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.338Z: Expanding GroupByKey operations into optimizable parts.
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.386Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.504Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.544Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.582Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:06.622Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:07.045Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 7:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:07.137Z: Starting 5 workers in us-central1-a...
May 14, 2020 7:02:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:34.829Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 14, 2020 7:02:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:34.867Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 14, 2020 7:02:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T19:02:35.181Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 14, 2020 7:02:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:40.234Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 14, 2020 7:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:52.402Z: Workers have started successfully.
May 14, 2020 7:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:02:52.450Z: Workers have started successfully.
May 14, 2020 7:03:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:03:26.114Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 7:03:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:03:26.376Z: Cleaning up.
May 14, 2020 7:03:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:03:26.469Z: Stopping worker pool...
May 14, 2020 7:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:05:32.504Z: Autoscaling: Resized worker pool from 5 to 0.
May 14, 2020 7:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T19:05:32.556Z: Worker pool stopped.
May 14, 2020 7:05:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-14_12_01_57-15504423999323501362 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e39701ee-20f4-4eb9-a1fb-2e7ae6b4cd55 and timestamp: 2020-05-14T19:05:38.679000000Z:
Metric: Value:
read_time 17.158
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 7:05:39 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 12 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 3.091 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 58s
103 actionable tasks: 98 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/kjoa2zswjk2du
Stopped 11 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #502
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/502/display/redirect>
Changes:
------------------------------------------
[...truncated 293.42 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 14, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-HKxZicn-QC8sTsiTpS1t_w.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-KROGTqm6Rw3mx0i4UmJFLw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-gu_gs17I_w7Vt7O7zNXILw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests--Hp46d_HeiEAlqAegz36bw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-eWrkRyyfFKNmSu0YIGVakw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-9CQmOX5Me3mNtL3R5XAToA.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-IQNEyXgPmdHLhxcFqZcIUw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-riaZSfWPvmIj67oFu3FXww.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ifu29u0MYoj9itdzZ4Wkfw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-nEANfjaihptzrF_KsPbcEw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Mw5D0_fY_6FFhwbI1nYFtg.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-23rSctrzZxN4-Rj4DA6TDg.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-sOLA6JazYq5UBDO1EjVc1g.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-EvuVgb4oxx8r5xIEcxpJFw.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-q1EhzGBOylPn4BTZ8_NJMg.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-eV-btAa0DxiTEo5em98QyA.jar
May 14, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-mQGd6imZGC8jOjzKG6EOQg.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-EBjzyRxM_06fEdx7kqAzkw.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-AE1qNr7bajt2gXFgv5Tthg.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-4c5n7A3SKv78f0QANz946g.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-qUV8DPgV7sQEmbEO_adN2g.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-VAv37hb4V2W2jEC_c8LYNg.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-sQKDjrDQSxssPPai2N6khw.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-GQLvFZRVNcqhroFZJzbxJA.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-19RUqCEfQZV2djJbTJioKw.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-IkVZM_8UvU6g41dkRHQYLQ.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-IfkdXYdhqKdG5TnX_nEd4g.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-QEftjO055e1z8YDDira0fA.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7sOzqvyYcx9-QpPPscygTw.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-g3zJKYCRloAaVfzv1zSELA.jar
May 14, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7sOzqvyYcx9-QpPPscygTw.jar
May 14, 2020 12:45:30 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7sOzqvyYcx9-QpPPscygTw.jar&uploadType=resumable&upload_id=AAANsUlOh4GlF268TLiUwO1QRdfYHFAIKL7U1Y4nKQ9LGssD8zSSpDanXpz9kfyIGOG0UiJ_a5HpDqvPwjc0UMbXufFLTS9ctw.
May 14, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 14, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7sOzqvyYcx9-QpPPscygTw.jar
May 14, 2020 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 85OYWWo3jl6IBBU8WBxkOQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-85OYWWo3jl6IBBU8WBxkOQ.pb
May 14, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 14, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-14_05_45_36-878884159274252502?project=apache-beam-testing
May 14, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-14_05_45_36-878884159274252502
May 14, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-14_05_45_36-878884159274252502
May 14, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T12:45:37.019Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 14, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:41.438Z: Checking permissions granted to controller Service Account.
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:46.081Z: Worker configuration: n1-standard-1 in us-central1-a.
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:46.909Z: Expanding CoGroupByKey operations into optimizable parts.
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.000Z: Expanding GroupByKey operations into optimizable parts.
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.044Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.310Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.371Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.424Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 14, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:47.493Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 14, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:48.014Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:45:48.105Z: Starting 5 workers in us-central1-a...
May 14, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T12:45:56.487Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 14, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:46:26.053Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 14, 2020 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:46:41.885Z: Workers have started successfully.
May 14, 2020 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:46:41.910Z: Workers have started successfully.
May 14, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:47:17.175Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:47:17.524Z: Cleaning up.
May 14, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:47:17.709Z: Stopping worker pool...
May 14, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:49:18.691Z: Autoscaling: Resized worker pool from 5 to 0.
May 14, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T12:49:19.041Z: Worker pool stopped.
May 14, 2020 12:49:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-14_05_45_36-878884159274252502 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f2e7edac-290d-47c3-bc08-835626fab380 and timestamp: 2020-05-14T12:49:25.264000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.015
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 12:49:25 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 4 mins 6.419 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 8s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/y7tivj7edtfss
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #501
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/501/display/redirect?page=changes>
Changes:
[github] [BEAM-9928] Add qualified import name to task descriptors (#11690)
------------------------------------------
[...truncated 293.91 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 14, 2020 6:45:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-NIbX1n8Tw98B9pFXc3-12w.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-VCQx5AV0JvUuug3V0K-jcw.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-OV2NWbTUyGRy_mqKff0uTA.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-pOXM8mT05CEMIsM0BVq8kg.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-s4NqjEzIlpZxJDY8vn19Pw.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--X2cv9r1n27uoxirFvFRuw.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-byDJ48Wlj_X3zWwOQEhzJA.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-VPxbr_dgIaQfoItkmjdFYg.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-UPN-rY65T6EVc3laqDLW1A.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-74JDIbBv9WYoL_k-oK71yw.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-9h0gSzCah3u459ciP7FjAg.jar
May 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-6aXEHrghpqCtYOMpDO0H6w.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-FJSiMPxDe-7gqQFHRPkclg.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MYvXZI3Q8WECpY-6iMb9EA.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-PlLy3GHP4fry4CC6FoVjEg.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-TRFAbsv3XNSaiJjSsHElSQ.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wzg_Hv1Oyo0EPUCtjstu-A.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-kLVzttmkgrOeJ9Chyx4jNA.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-1UeCSeDRIiTj0NsSLr_VmA.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-kSz1Tzj3IaowC_n-1k6Nvw.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-9YC2ou0TLhyL3vWVmEfgcA.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-aISu8NsiJuOcv4yqlvj-ag.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-2srsKJWF2p1JxX605ek6Jg.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-f4JnNEoqCKJCNi8U6ETQnw.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Q92b5ZfKWL-gmdngmV2eFw.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QS1XbnR_Ujg4Xh3Tpnfx9g.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-O8t6qxaAH-uhAtD8ycxAuw.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Arr5wZuthmAg971WL9OA9Q.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Fc-qQnuTl0gDcyLW3iYnRA.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Uje1p99zWfQjUVdOATX9OQ.jar
May 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QS1XbnR_Ujg4Xh3Tpnfx9g.jar
May 14, 2020 6:45:38 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QS1XbnR_Ujg4Xh3Tpnfx9g.jar&uploadType=resumable&upload_id=AAANsUnTK1pa4lFuj0s1kssoIdGeHJx8MYHlJvHewWiW13O7gMhgzH9tDMu9px5Q56-mQqX_YOyW6PwuXh9m9BhUh_A.
May 14, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 14, 2020 6:45:38 AM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
May 14, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-QS1XbnR_Ujg4Xh3Tpnfx9g.jar
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 5s9oigNPl159VAdXVT1WQw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5s9oigNPl159VAdXVT1WQw.pb
May 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 14, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_23_45_46-8138915856282765521?project=apache-beam-testing
May 14, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-13_23_45_46-8138915856282765521
May 14, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-13_23_45_46-8138915856282765521
May 14, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T06:45:46.347Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 14, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:50.603Z: Checking permissions granted to controller Service Account.
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:55.272Z: Worker configuration: n1-standard-1 in us-central1-a.
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:55.960Z: Expanding CoGroupByKey operations into optimizable parts.
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.001Z: Expanding GroupByKey operations into optimizable parts.
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.033Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.117Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.153Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.178Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.250Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.595Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:45:56.683Z: Starting 5 workers in us-central1-a...
May 14, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T06:46:01.710Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 14, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:46:24.950Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 14, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:46:24.992Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 14, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:46:30.361Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 14, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:46:50.129Z: Workers have started successfully.
May 14, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:46:50.154Z: Workers have started successfully.
May 14, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:47:21.237Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:47:21.445Z: Cleaning up.
May 14, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:47:21.523Z: Stopping worker pool...
May 14, 2020 6:49:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:49:20.224Z: Autoscaling: Resized worker pool from 5 to 0.
May 14, 2020 6:49:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T06:49:20.371Z: Worker pool stopped.
May 14, 2020 6:49:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-13_23_45_46-8138915856282765521 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 336263a6-c432-4e89-bf30-5fedb6651037 and timestamp: 2020-05-14T06:49:25.880000000Z:
Metric: Value:
read_time 13.681
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 6:49:26 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 59.658 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 8s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/jctwgzjr6xv7a
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #500
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/500/display/redirect?page=changes>
Changes:
[sniemitz] Allow users of AvroIO to specify a custom DatumReader implementation
[tysonjh] Disambiguate method filter for toArray.
[pabloem] Revert "Merge pull request #11673 from [BEAM-9967] Adding support for BQ
[thw] Bump Flink 1.10 version
[kcweaver] [BEAM-9975] Log invalid dictionary after unsuccessful struct conversion.
[github] [BEAM-9945] Ensure that the read index represents the number of fully
[github] [BEAM-9942] reduce groovy code duplication in python test-suites
[boyuanz] [BEAM-9935] Respect allowed split points in Java
[github] Switch to %s
[github] [BEAM-9001, BEAM-6327] Ensure that all transforms (except for required
------------------------------------------
[...truncated 298.58 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 1:20:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 1:20:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 14, 2020 1:20:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 14, 2020 1:20:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 14, 2020 1:20:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 14, 2020 1:20:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-jHxjS_L9R9VxsJUU5ofO9g.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-K4cZy4hORz9pCOIKlaWi7w.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-yKwWCOdFa5907LITQHEsdQ.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Mwu0LcrwaApPtJk-7PL0rQ.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-da6Y69d3sEUVxOulE6mwOw.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-wqZgz066Sdf8Jtdp20YZtA.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-fEdfyamG-WFWD-dAFmx-BQ.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-5XLyn551ypprmTkb8wLshA.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-7KH9DR0LoQPUcJy7cCLC4w.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-AXE7Y0NHFCHxr-3tCxI11A.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-yRWfQXPLTXoO0WyciJn8AA.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-usKNKP8cdR-lhBqlB1TlDw.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-2bwpq6Gl7VoJCfODmwDIpQ.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-jodLeOBJHu0QhiwBLsnl0Q.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-J73b42Iy36vuHhMP782pbg.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-PZhrKlDWHhn807PNA5rgQQ.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-_SZIn6peb1uCfcVU28idvg.jar
May 14, 2020 1:20:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Yn51-GgQubKhJB9bRe5G5Q.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-vLMoUxuZAvVjj--HDDK4jw.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-xQmV7alCQ94zCSR_CPvT0g.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-LCyLknnPgBINM7K8rmM9bA.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-oj1_io_nj97o-xuMuiibNw.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-EU0i6hIpMZ9tILvIai11bQ.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-J_XBV55_Jc9Yu6FVU_G07Q.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-7sXS6lMI6Ybypq3ARQERHA.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-YswHzyzDZJmUwx-iW28Z6Q.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-hVCIJ7PQUg1dVpBR-bTE6w.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Z0-Zb1C3sQ3b7zrjBE8MUw.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6LRVoM8rF9Pehf-m70ftRg.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-1Q-BhFWhVg1m_YJxHVJnOg.jar
May 14, 2020 1:20:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6LRVoM8rF9Pehf-m70ftRg.jar
May 14, 2020 1:20:38 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6LRVoM8rF9Pehf-m70ftRg.jar&uploadType=resumable&upload_id=AAANsUm-kCfdAd4VniRYGVrdUG945Ldiu3-hOr3KwTObLArxZ4Mru0AEm_hBfhKhoY_3kPet2MlmTqc7QDQyVnLa2l9mhE5J_Q.
May 14, 2020 1:20:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 14, 2020 1:20:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-6LRVoM8rF9Pehf-m70ftRg.jar
May 14, 2020 1:20:45 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 10 seconds
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 14, 2020 1:20:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12104 bytes, hash hfWit6AZ8wxMMmJwx9YuLw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-hfWit6AZ8wxMMmJwx9YuLw.pb
May 14, 2020 1:20:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 14, 2020 1:20:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_18_20_47-12760501419027389557?project=apache-beam-testing
May 14, 2020 1:20:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-13_18_20_47-12760501419027389557
May 14, 2020 1:20:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-13_18_20_47-12760501419027389557
May 14, 2020 1:20:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T01:20:47.204Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 14, 2020 1:20:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:51.415Z: Checking permissions granted to controller Service Account.
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:56.192Z: Worker configuration: n1-standard-1 in us-central1-a.
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:56.948Z: Expanding CoGroupByKey operations into optimizable parts.
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:56.985Z: Expanding GroupByKey operations into optimizable parts.
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.026Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.109Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.141Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.179Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.220Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.656Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 1:20:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:20:57.752Z: Starting 5 workers in us-central1-a...
May 14, 2020 1:21:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T01:21:14.410Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 14, 2020 1:21:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:21:23.124Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 14, 2020 1:21:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:21:23.156Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 14, 2020 1:21:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:21:28.517Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 14, 2020 1:21:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:21:41.622Z: Workers have started successfully.
May 14, 2020 1:21:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:21:41.654Z: Workers have started successfully.
May 14, 2020 1:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:22:21.419Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 14, 2020 1:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:22:21.632Z: Cleaning up.
May 14, 2020 1:22:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:22:21.737Z: Stopping worker pool...
May 14, 2020 1:24:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:24:23.742Z: Autoscaling: Resized worker pool from 5 to 0.
May 14, 2020 1:24:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T01:24:23.795Z: Worker pool stopped.
May 14, 2020 1:24:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-13_18_20_47-12760501419027389557 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 4775ecb0-69dc-42c2-a283-6258fb786ae8 and timestamp: 2020-05-14T01:24:30.893000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.759
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 14, 2020 1:24:31 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 4 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 7.048 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 35s
103 actionable tasks: 78 executed, 25 from cache
Publishing build scan...
https://gradle.com/s/tjvu7l7ibstkm
Stopped 3 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #499
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/499/display/redirect?page=changes>
Changes:
[kawaigin] Use csv reader instead of split to read csv data.
------------------------------------------
[...truncated 293.24 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 13, 2020 6:59:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-bLdLAtKDPzZd3YV0ujKkWg.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-a4dn6xDcuA5lC-rdbb41aA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-a-fxjsmeubbNHZKojrTyyA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-OFBMpnUcMRRV9iFHTwcoFA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-iO5gSLJ9Ll4wyrxdPszHCQ.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-l3Kobk7Ror2Bt_Bvf0imWg.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-iRZ_lfsM4QLNhFgXEsdCNA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wtkNLdeS5yZvhEhtvu_gew.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Vsjbbndra2lvlM2aAxHTAA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-S8H4OtwrDy6YpM2_Wz2h2A.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-sUvsaWdWiBmpeYzjEpo9-w.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-36YZ4xFeXti18qd79T0NCg.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Grkhx1DsdInlZg6Rb-RRsA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-pthQjND1bqixUxdVuGYF7Q.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_X8WJeyFauMpa6mAwOE-FQ.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-tLJhipMyUfRzJ3Uag_tvnw.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-vOxJWWFYPrDu46I-_HkIXg.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Cez4elisfGkWdUOIhGd-BQ.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-zgj1YjO3wxT-i-Pr8NzCaA.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-_h0e2AMmC-2irGerpcnqwQ.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-mfYyZGI2q3_v3P-AmmVArw.jar
May 13, 2020 6:59:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-U9QqTYWXFZy1cAcbLiv-wQ.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-WslIKU7uiANAKNF40Pi43A.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-l8krmfI4f1JiAwX7mXUZHQ.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-zc27yCAo7fPR6tdHaDgpQA.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-1fWjcMAtzTYs1PUnOwsgxA.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-R9EYA2RpQ77898h3PPV18w.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-CK3SI0FxIHP-sn0GSqxKvw.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-F4QGQiYmQfznCbmSey0IDw.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-CK3SI0FxIHP-sn0GSqxKvw.jar
May 13, 2020 6:59:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-IzCzWaeMOO_UGh-BydMqMQ.jar
May 13, 2020 6:59:15 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-CK3SI0FxIHP-sn0GSqxKvw.jar&uploadType=resumable&upload_id=AAANsUkbcoarDDPl0fuUeEqub2n8vicUSi_38l4oZxyRiWcLcGxKJ8olqKKf4fUAi1PZeJTVErN1_h16yUBwnryKcCw.
May 13, 2020 6:59:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 13, 2020 6:59:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-CK3SI0FxIHP-sn0GSqxKvw.jar
May 13, 2020 6:59:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 13, 2020 6:59:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12102 bytes, hash ns9NsH2tuaXq68DICZqgkw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ns9NsH2tuaXq68DICZqgkw.pb
May 13, 2020 6:59:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 13, 2020 6:59:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_11_59_20-5279563333764980321?project=apache-beam-testing
May 13, 2020 6:59:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-13_11_59_20-5279563333764980321
May 13, 2020 6:59:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-13_11_59_20-5279563333764980321
May 13, 2020 6:59:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T18:59:20.967Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 13, 2020 6:59:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:25.511Z: Checking permissions granted to controller Service Account.
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:30.309Z: Worker configuration: n1-standard-1 in us-central1-a.
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:30.946Z: Expanding CoGroupByKey operations into optimizable parts.
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:30.993Z: Expanding GroupByKey operations into optimizable parts.
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.021Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.153Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.188Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 13, 2020 6:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.221Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 13, 2020 6:59:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.260Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 13, 2020 6:59:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.654Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 6:59:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:31.735Z: Starting 5 workers in us-central1-a...
May 13, 2020 6:59:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T18:59:50.626Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 13, 2020 6:59:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T18:59:57.415Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 13, 2020 7:00:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:00:14.533Z: Workers have started successfully.
May 13, 2020 7:00:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:00:14.557Z: Workers have started successfully.
May 13, 2020 7:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:00:51.017Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 7:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:00:51.240Z: Cleaning up.
May 13, 2020 7:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:00:51.344Z: Stopping worker pool...
May 13, 2020 7:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:02:42.870Z: Autoscaling: Resized worker pool from 5 to 0.
May 13, 2020 7:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T19:02:42.935Z: Worker pool stopped.
May 13, 2020 7:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-13_11_59_20-5279563333764980321 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 73bada58-29e0-4003-a27f-eaf6315b29d7 and timestamp: 2020-05-13T19:02:56.142000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.34
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 7:02:56 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 53.186 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 55s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ofagjm4u7n3wk
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #498
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/498/display/redirect>
Changes:
------------------------------------------
[...truncated 293.78 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 13, 2020 1:03:29 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 13, 2020 1:03:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 13, 2020 1:03:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-S1BDiuEEcJ-3thJO5o2M5Q.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-2rA5pC5sQH7jv6Zm9WtZHA.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-JEXMsItWqS918IZJP8y0Ng.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-zQu6aX31pEvle8dbbfSPng.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-5dRHQUWBNkZg0gs3RLaGLg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-n5DOYmRpefhcfsZAD41Suw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-Xez0JeodNSQ8yIGiOgmdGg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-lRAFD1xpFXfiQLT72ibT9Q.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-pXZ2QxDZOVVF0zkQnIS5ng.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-esLr2HkrsakrfLyOIbny9g.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-R-78x-5MwM-pWto1iOwTBw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-sIUenmTek9rencormnSXBw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-iiNWW9svxfXDR55FxTTrUQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-mqQl_XFQsSmDOL1PBM31sw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-hjKdL5aSAputzJwccbMhPg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-bxy04-UbbzjkaC0E63oU3Q.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-gERiwU-n2WS9G1_Yw3gFwg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-9P28IoJwlw6C8PbMR9pgMQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-EubdZvDu9bd7fKbIjq6XYg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-AaqZAkwmUGrNyeKkPG25QQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-amHtlD4ZkwpwbpA6T31QMA.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-IYThpxLsZpLwpazVq1k3sw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-L7HaOEDgmEsSUS1FsXyk0w.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-GlS_ph0nSrjvhmG5fBQnUQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-PymJJdTpgjlmHQJeOavxtQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-P1ypt-0k0KeYPU4CnXt7Fw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-7gZXBiWvpiZEVCaeraFyMg.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ufCs-ac-dEVxmOPtap6I2A.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-o1eSvQ9k4plncYaky4b-Qw.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-6pHhxCFprKmSaNKJ4enWWQ.jar
May 13, 2020 1:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ufCs-ac-dEVxmOPtap6I2A.jar
May 13, 2020 1:03:33 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ufCs-ac-dEVxmOPtap6I2A.jar&uploadType=resumable&upload_id=AAANsUnmHQqZhpIIl_5SOPQx4ru5ZitFuJoB24tZR1-VSN1EmPknVkMjWiauWD2Qz8s8mvpOBo6NJD9GBWWnKcKFAHvOOnHFAg.
May 13, 2020 1:03:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 13, 2020 1:03:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ufCs-ac-dEVxmOPtap6I2A.jar
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 13, 2020 1:03:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 5Zxrj8j87lfHWDGpFNgf6A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5Zxrj8j87lfHWDGpFNgf6A.pb
May 13, 2020 1:03:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 13, 2020 1:03:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-13_06_03_39-1309883624251848341?project=apache-beam-testing
May 13, 2020 1:03:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-13_06_03_39-1309883624251848341
May 13, 2020 1:03:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-13_06_03_39-1309883624251848341
May 13, 2020 1:03:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T13:03:39.288Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 13, 2020 1:03:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:42.797Z: Checking permissions granted to controller Service Account.
May 13, 2020 1:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:46.815Z: Worker configuration: n1-standard-1 in us-central1-f.
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.496Z: Expanding CoGroupByKey operations into optimizable parts.
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.527Z: Expanding GroupByKey operations into optimizable parts.
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.561Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.643Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.676Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.710Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:47.748Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:48.119Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 1:03:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:03:48.211Z: Starting 5 workers in us-central1-f...
May 13, 2020 1:04:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T13:04:01.198Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 13, 2020 1:04:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:04:12.177Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 13, 2020 1:04:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:04:12.215Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 13, 2020 1:04:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:04:17.843Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 13, 2020 1:04:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:04:37.014Z: Workers have started successfully.
May 13, 2020 1:04:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:04:37.045Z: Workers have started successfully.
May 13, 2020 1:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:05:10.891Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 1:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:05:11.095Z: Cleaning up.
May 13, 2020 1:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:05:11.199Z: Stopping worker pool...
May 13, 2020 1:06:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:06:57.906Z: Autoscaling: Resized worker pool from 5 to 0.
May 13, 2020 1:06:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T13:06:57.968Z: Worker pool stopped.
May 13, 2020 1:07:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-13_06_03_39-1309883624251848341 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2d9b59f7-0cd2-46f1-9b25-cde5b321790f and timestamp: 2020-05-13T13:07:05.713000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.839
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 1:07:06 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 45.493 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/jb2hkj7i2ncgc
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #497
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/497/display/redirect?page=changes>
Changes:
[github] [BEAM-9959] Root Transform fixes (#11686)
------------------------------------------
[...truncated 293.95 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 6:51:19 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 6:51:19 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 13, 2020 6:51:19 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 6:51:19 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 13, 2020 6:51:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 13, 2020 6:51:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 13, 2020 6:51:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 13, 2020 6:51:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-NlrhH6DoDXeD0ZDUvu8CBw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Njxup27PyjS9uYzUvnAM6A.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-KExX5ezlafROgcIeFZp3pw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-eHOnflxtYYU0SToMrGymiw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-vc91HHutt_I7O2xMbhnFeA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--xvGud0mx9M_bWA6MSXsUw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-OnatwDW2AnwAruzG6NJwFA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-_U0b0LQUcfEDWj2T5-VEMA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ihhA1NsEDVkU0LrjzRS2hg.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-xFb3Xf-geZRYcR6e9Lk7KA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6YUQkxOzuyv1eus_X3b9hw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-qfxTUF2z64UgrV7DHz3upQ.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-YZaEcbw-ahLaSnBu1G8Kgw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-m07dJv5t_U-wld7N5GxkaA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-74U8PrX7XNuCnf17Z8FoXQ.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT--BWAv3Qni6U7ZBhCW0xeJA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-vlgunMUcPbSBe1lB_QvNMg.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-tGems8Y_ejGw0teis-TtkA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-7Py9vMLQqHGD2jgXFJQkFw.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-R5aKxUOKCCjK6AcCxFLZOQ.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-JHk166QwBRtzG4e3TY8_QA.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-cQ8-HerN05XlD1UmGLvD9A.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-12lS-80BjOnO4g2JwxOWgQ.jar
May 13, 2020 6:51:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-w0rlQiuP_U2SLTbEUV7n5g.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-98HEkCe7kuU706850MgWhg.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-eQ8n3DH9Yqjz_RFG3d5ulQ.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-J1OJe4lw8AOFPzXgwn1LOA.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-1tUsOws5FHt3nM5vML-iCw.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-MhCf3IGunEnVa-B0UAxYMw.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8AB6XtZINEOobxcdsYCCIw.jar
May 13, 2020 6:51:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8AB6XtZINEOobxcdsYCCIw.jar
May 13, 2020 6:51:23 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8AB6XtZINEOobxcdsYCCIw.jar&uploadType=resumable&upload_id=AAANsUlClC2d3UD-aJ2MDaCwsWUr4uqU_ozB4M21rHhi_L1Wx7CScMk8W1BbQjGqi6ZJfa43p-bYpDDLVSVuLiK3eJVX0ZpzZw.
May 13, 2020 6:51:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 13, 2020 6:51:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8AB6XtZINEOobxcdsYCCIw.jar
May 13, 2020 6:51:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 13, 2020 6:51:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 13, 2020 6:51:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 13, 2020 6:51:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 13, 2020 6:51:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 13, 2020 6:51:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 13, 2020 6:51:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12102 bytes, hash LFmfE-4wkWH0WTqRSrrPJg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LFmfE-4wkWH0WTqRSrrPJg.pb
May 13, 2020 6:51:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 13, 2020 6:51:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_23_51_28-2499968646247572212?project=apache-beam-testing
May 13, 2020 6:51:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-12_23_51_28-2499968646247572212
May 13, 2020 6:51:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-12_23_51_28-2499968646247572212
May 13, 2020 6:51:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T06:51:28.760Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 13, 2020 6:51:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:32.023Z: Checking permissions granted to controller Service Account.
May 13, 2020 6:51:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:36.947Z: Worker configuration: n1-standard-1 in us-central1-f.
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.585Z: Expanding CoGroupByKey operations into optimizable parts.
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.619Z: Expanding GroupByKey operations into optimizable parts.
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.645Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.733Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.774Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.811Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:37.846Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:38.294Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 6:51:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:51:38.379Z: Starting 5 workers in us-central1-f...
May 13, 2020 6:51:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T06:51:55.032Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 13, 2020 6:52:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:52:06.336Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 13, 2020 6:52:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:52:06.374Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 13, 2020 6:52:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:52:11.764Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 13, 2020 6:52:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:52:26.703Z: Workers have started successfully.
May 13, 2020 6:52:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:52:26.738Z: Workers have started successfully.
May 13, 2020 6:53:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:53:06.710Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 6:53:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:53:06.946Z: Cleaning up.
May 13, 2020 6:53:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:53:07.092Z: Stopping worker pool...
May 13, 2020 6:54:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:54:41.128Z: Autoscaling: Resized worker pool from 5 to 0.
May 13, 2020 6:54:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T06:54:41.183Z: Worker pool stopped.
May 13, 2020 6:54:47 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-12_23_51_28-2499968646247572212 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f46ec487-19c5-483f-bb32-f8b97974d31e and timestamp: 2020-05-13T06:54:47.375000000Z:
Metric: Value:
read_time 22.707
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 6:54:47 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 35.456 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 38s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/5el3cp7lqcpf2
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #496
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/496/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9164] Re-enable UnboundedSourceWrapper#testWatermarkEmission test
[github] [BEAM-9622] Add Python SqlTransform test that joins tagged PCollections
------------------------------------------
[...truncated 294.19 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 12:57:56 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 12:57:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 13, 2020 12:57:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 13, 2020 12:57:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 13, 2020 12:57:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 13, 2020 12:57:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-kc6BqZEUjN06x24QnnNW4A.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-axeH0ovxXLrXW4hT9Xkykw.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-6Ns3S0yrJUCR-WZWUZe73A.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-cev1RZ_FubihA1zqvxA2LQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-E1ZipbwcCHa4L0OnJUr8aQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-IDHcQkU7Tdmt1crVX0h5SQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-_t-1ZZy1Vb4zXS6KcKc1Lw.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-5F4YeGDstPtLWdeuCxTO3w.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-aMB6SF0xuWGoD8mdD4qHSQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-VgWLzl-WWT7pG0myEKkRHw.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ZRjAsKRHN7qZKEy7X-dWng.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-qnn6tPIGXMXWwD4I5wqxRA.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-WHjWO0s78aWzuHC20KsV3A.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-kV_2oPQ3RBJk0LFDGwFFAQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-XutTtrRE9noXGKgLrgTBRQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-9FuGayONWVSzwDRnieNf3w.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Go0feCY1hZvvrT1GLoEgRQ.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-aX9YXrJXFnolfrXDrheKxg.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-a1V0qb_N7fm_0gJoJC5JVw.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-eQzEOuryw7ySGPiP63Uolw.jar
May 13, 2020 12:57:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-saiQcwchb3prMb0xOkbc9Q.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-DxKCPXDSbmwh3suaB_KpOA.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-fYwCrtIuU6wHPncp2xYwAw.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-KpXZDA6xkWntKVim6dKc_Q.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_nZQEGJveqpMlPW8U6egVQ.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JIve_yk1f9W7YwwXsdcivg.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_nZQEGJveqpMlPW8U6egVQ.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-HKGDm4mMHSOh6P1JYXza9g.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-TzRgatYURklIMnrM8YyTMw.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-SyS68HYXLKSOCOf3EBghmg.jar
May 13, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-o-um0IHUJP4vfzc8A0uClg.jar
May 13, 2020 12:58:00 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_nZQEGJveqpMlPW8U6egVQ.jar&uploadType=resumable&upload_id=AAANsUnsAUgGvDltsQ7mLUR2B8kigJXBzKMi3SGpZKrV3FAXzH9W8xy401uu-vpV2dc7sAlfu0ZXGdPasKNRUeSTOkU.
May 13, 2020 12:58:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 13, 2020 12:58:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_nZQEGJveqpMlPW8U6egVQ.jar
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 13, 2020 12:58:07 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash pNHwKSynkL7I-aR8wX2U6g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pNHwKSynkL7I-aR8wX2U6g.pb
May 13, 2020 12:58:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 13, 2020 12:58:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_17_58_08-4137528138717642632?project=apache-beam-testing
May 13, 2020 12:58:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-12_17_58_08-4137528138717642632
May 13, 2020 12:58:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-12_17_58_08-4137528138717642632
May 13, 2020 12:58:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T00:58:08.337Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 13, 2020 12:58:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:11.319Z: Checking permissions granted to controller Service Account.
May 13, 2020 12:58:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:15.284Z: Worker configuration: n1-standard-1 in us-central1-a.
May 13, 2020 12:58:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:15.956Z: Expanding CoGroupByKey operations into optimizable parts.
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:15.999Z: Expanding GroupByKey operations into optimizable parts.
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.039Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.113Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.145Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.177Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.211Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.725Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 12:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:16.819Z: Starting 5 workers in us-central1-a...
May 13, 2020 12:58:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-13T00:58:27.322Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 13, 2020 12:58:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:58:42.123Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 13, 2020 12:59:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:59:03.846Z: Workers have started successfully.
May 13, 2020 12:59:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:59:03.877Z: Workers have started successfully.
May 13, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:59:35.572Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 13, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:59:35.782Z: Cleaning up.
May 13, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T00:59:35.889Z: Stopping worker pool...
May 13, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T01:01:21.337Z: Autoscaling: Resized worker pool from 5 to 0.
May 13, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-13T01:01:21.387Z: Worker pool stopped.
May 13, 2020 1:01:28 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-12_17_58_08-4137528138717642632 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c67a90b1-669b-4c81-a45b-470db0dae5ce and timestamp: 2020-05-13T01:01:28.940000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.642
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 13, 2020 1:01:29 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 40.464 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/pxbbsfysbsj3a
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #495
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/495/display/redirect?page=changes>
Changes:
[kamil.wasilewski] Fix InfluxDB measurement names to match those in Grafana dashboards
[github] [BEAM-9945] Report data channel progress via a designated counter.
[github] [BEAM-9577] Update Java Runners to handle dependency-based artifact
[github] Merge pull request #11673 from [BEAM-9967] Adding support for BQ labels
------------------------------------------
[...truncated 297.20 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 6:49:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 6:49:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 12, 2020 6:49:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 6:49:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 12, 2020 6:49:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 12, 2020 6:49:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 12, 2020 6:49:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 12, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-4CV_LIvmTqH6UwpOTxCs2g.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ZxL4DoOxFvjRQBMqizaeYQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-uNiElGorWaQQxGLEGnm0qg.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-11R-ZdFvUHeQI5p_esposA.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-szZdPhVt4zff4_6dyOynkw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-QFe2wGFm0bkQwrXIP15z6w.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-F1B9hhdDNfFHjlQbIVqMjA.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-4evuzLTUbHyLpqCTZGr6oQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-gmD1p_PL46At_Hn2gYsIoA.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-7QxcWw2SdcJDf3xijx_cVA.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-IJXBbYaCPR6qAtEqUoyreQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-Se6exkMtDLLiXX9Lt5dhHw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-LgYwa__AOdTc7wZpIe35Pw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-OomXRlk6ovUaF0_6y_Lh3g.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-xIlYtRf27QXe9FSiHxpSGw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-VNVUfZ3-6iKrh3dO37Y5Dw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-2KaB7z3yX0PCymqv_aJWMA.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-UTmlIaw15a0Xx15AWZfkPQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-e1TBAlKdzDru-nf6Jg7JUw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Dp5C70PfT17ssGVGGNfApw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-uuXXpcad8HQjPLDNuOuYUw.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-qpvJUgMOs9fU7uUrIZtFWQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-e3lrgnOeAcMDpOdT535KJQ.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-rBRYOAcvNrHClX_O5mv1zg.jar
May 12, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Ge3zU5a7mQE8U_gHdvnMHg.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-mDF-xJSyxEQYWt2SIq2YMw.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-bow_ZK4ekdp4QwuBHVHLoA.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-AfEVD5mhBsLPDd3rQS1x-w.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-dK424xoZHIEn1MF_Gq9zdA.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-VeHTz2G-a0Uusz7q7CjsBw.jar
May 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-dK424xoZHIEn1MF_Gq9zdA.jar
May 12, 2020 6:49:14 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-dK424xoZHIEn1MF_Gq9zdA.jar&uploadType=resumable&upload_id=AAANsUmMPjz5FkSufmvAjGFtwTn3Zv6m3zdPaz8EF-dQ1ZtX0vjWNCPz1Tf2hm66FkJ-PkXkvF4l7KP26-eFV4RFZioWxdUqQg.
May 12, 2020 6:49:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 12, 2020 6:49:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-dK424xoZHIEn1MF_Gq9zdA.jar
May 12, 2020 6:49:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 16 seconds
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 0l78a_wSETYJW1Hp9iU-bw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0l78a_wSETYJW1Hp9iU-bw.pb
May 12, 2020 6:49:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 12, 2020 6:49:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_11_49_29-17416091792970954561?project=apache-beam-testing
May 12, 2020 6:49:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-12_11_49_29-17416091792970954561
May 12, 2020 6:49:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-12_11_49_29-17416091792970954561
May 12, 2020 6:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T18:49:29.937Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 12, 2020 6:49:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:33.124Z: Checking permissions granted to controller Service Account.
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:37.237Z: Worker configuration: n1-standard-1 in us-central1-a.
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:37.967Z: Expanding CoGroupByKey operations into optimizable parts.
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.010Z: Expanding GroupByKey operations into optimizable parts.
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.040Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.136Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 12, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.166Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 12, 2020 6:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.228Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 12, 2020 6:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.262Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 12, 2020 6:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.726Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 6:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:49:38.799Z: Starting 5 workers in us-central1-a...
May 12, 2020 6:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:50:05.328Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 12, 2020 6:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:50:05.355Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 12, 2020 6:50:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T18:50:07.684Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 12, 2020 6:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:50:10.726Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 12, 2020 6:50:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:50:23.397Z: Workers have started successfully.
May 12, 2020 6:50:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:50:23.426Z: Workers have started successfully.
May 12, 2020 6:51:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:51:10.680Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 6:51:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:51:10.933Z: Cleaning up.
May 12, 2020 6:51:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:51:11.007Z: Stopping worker pool...
May 12, 2020 6:53:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:53:10.923Z: Autoscaling: Resized worker pool from 5 to 0.
May 12, 2020 6:53:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T18:53:10.968Z: Worker pool stopped.
May 12, 2020 6:53:21 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-12_11_49_29-17416091792970954561 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7bb0a7df-f99f-4537-bf7f-83e80e727e84 and timestamp: 2020-05-12T18:53:21.277000000Z:
Metric: Value:
fields_read 4375276.0
read_time 27.069
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 6:53:21 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4 mins 18.901 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 40s
103 actionable tasks: 72 executed, 31 from cache
Publishing build scan...
https://gradle.com/s/arkvkt4o3jrig
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #494
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/494/display/redirect?page=changes>
Changes:
[kamil.wasilewski] [BEAM-6710] Add landing page with links to relevant dashboards
[mxm] [BEAM-9930] Add image to Beam Summit blog post
[mxm] [BEAM-9947] Store updated ParDoPayload for length-prefixed timer coders
[mxm] [BEAM-9947] Add unit test to check length prefixing of timer key
[kamil.wasilewski] Fix metrics reporting in HDFS IO IT tests
[mxm] [BEAM-9963] Fix coder type in failing ParDo streaming load test
[github] Merge pull request #11567: [BEAM-8132] Report Python metrics to InfluxDB
------------------------------------------
[...truncated 295.35 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 12, 2020 1:09:00 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 12, 2020 1:09:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 12, 2020 1:09:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-KU1j-KQK6xKthJaTaVe9Aw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-nfCpQYc0mf-XDZsdKDsn3Q.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-A1aUAu9e0zKiO2IBs8qlhQ.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-cOP8Dzl8noGzt8NK6F_JZw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-_Uk-SkHVea1ywamIPidl1A.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-JVeV0iVQ-EvQGKCAwF3XDQ.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-FFJR5Kz_qAUGoe7S6a3YOQ.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-qlKHSmDlZ7SyZAftB0j4cw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-J19K49mh_xXycsWG7Eme_w.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-LMHmvoSLjWyvEQQ6au7wlA.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-05Kpq5dTOJp6ZlyzWnbSig.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-S6SZD7dXg4GCOEXlAxB5eA.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-mMm1a9kf2imGj87wQ6DzyQ.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-PSX6HbN88qEyAfVZYjno0g.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-LJcDqa8Xi09MLxzKMlG55w.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-M2qxOymLqoFq3gR5tLMiRg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-RAHJ9XaGTuB64_yQMi4N0g.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-dsSZXVP8LZN2lQzXCZj6LA.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-fD5SiSEbWaQlwdg1pzgxSg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-sp1JW4l00wVO5QO7gdUUUg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kJ8Gcm2t77DbQhaMmWFNhg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vb6kzWNSc3Owc4gJUZ1bIg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-t1FbYogjm8EUqUY3Du4mrQ.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-qh3SqHUiyxvvqs9S-cjTcw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-PGI1uCig5yB2yM4SWLN_Bg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-S1mkr_QOqqhVPoSDS1qOOw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-oKs7obM5zuaHeYnmz1fa-g.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-q_-1YP_ifA82P5C0iSCBVw.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-lKfJKg4XPlxl7lf5Znigeg.jar
May 12, 2020 1:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-PWb3XRFNNMn4bmeaWvNrFw.jar
May 12, 2020 1:09:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vb6kzWNSc3Owc4gJUZ1bIg.jar
May 12, 2020 1:09:04 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vb6kzWNSc3Owc4gJUZ1bIg.jar&uploadType=resumable&upload_id=AAANsUkQHvl9XBozG_6kDR-eoYQeGJwfHmEioKjOKsweb1NQqtyBjKbbcbgJPuqtFIruo4y9lkYPQbtw_WoFqBsGKySEHzqo4w.
May 12, 2020 1:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 12, 2020 1:09:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vb6kzWNSc3Owc4gJUZ1bIg.jar
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 12, 2020 1:09:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash v1AGrHAoKd2pDYP5QXetRw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-v1AGrHAoKd2pDYP5QXetRw.pb
May 12, 2020 1:09:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 12, 2020 1:09:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-12_06_09_12-3142730858769860777?project=apache-beam-testing
May 12, 2020 1:09:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-12_06_09_12-3142730858769860777
May 12, 2020 1:09:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-12_06_09_12-3142730858769860777
May 12, 2020 1:09:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T13:09:12.584Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 12, 2020 1:09:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:17.388Z: Checking permissions granted to controller Service Account.
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:21.726Z: Worker configuration: n1-standard-1 in us-central1-a.
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.522Z: Expanding CoGroupByKey operations into optimizable parts.
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.559Z: Expanding GroupByKey operations into optimizable parts.
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.596Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.670Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.698Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.731Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:22.762Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:23.157Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 1:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:23.250Z: Starting 5 workers in us-central1-a...
May 12, 2020 1:09:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:51.123Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 12, 2020 1:09:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:51.151Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 12, 2020 1:09:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:09:56.538Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 12, 2020 1:09:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T13:09:56.684Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 12, 2020 1:10:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:10:11.150Z: Workers have started successfully.
May 12, 2020 1:10:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:10:11.179Z: Workers have started successfully.
May 12, 2020 1:10:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:10:45.295Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 1:10:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:10:45.518Z: Cleaning up.
May 12, 2020 1:10:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:10:45.608Z: Stopping worker pool...
May 12, 2020 1:13:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:13:25.486Z: Autoscaling: Resized worker pool from 5 to 0.
May 12, 2020 1:13:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T13:13:25.527Z: Worker pool stopped.
May 12, 2020 1:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-12_06_09_12-3142730858769860777 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 374b0b37-4e41-445b-b6e1-0bebaac340df and timestamp: 2020-05-12T13:13:30.782000000Z:
Metric: Value:
read_time 15.066
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 1:13:31 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 39.479 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 49s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/jf2cexqwiiq26
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #493
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/493/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-9642] Adding Go SDF fallback for unexpanded SDFs.
------------------------------------------
[...truncated 292.89 KB...]
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 12, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-1gAZSAWmezQhBcfi4DCjeQ.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-I7Mw3n7a_PhNzS5SpeaOpA.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-BbdaB7xmPaT_xxTmj8RI2g.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-MIST8TzcpXQBsZDap-T0FA.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-bjNuA3xMpDHhkZX5GcwY8g.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-PZdLLD7jUncAKVrq1RDQLg.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-G2Y18eCnA2G39rEXPxLlnQ.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-e_mX1T4as_oWfhX3F4tvmw.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-DG0I5QkyC0YKDdMKPlQGAQ.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-bYgVHzUd4fHvENQOQivUzA.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-WKX5ATtcIzXN8_WJenR7QA.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ZxvybltbiAW7aK2Pg5c7tw.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-2gVZdqHDtFo0gRk9O12C2g.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-np9ZjJM9xcLgJw2GZOQbrQ.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-_RlCMBuxlz-qRpOgR1581Q.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT--7JCnocw82CD1VLGZ3I1KQ.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-sysOeIlrDrHL3zFZIVuJNg.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-tizHpZfecLVTFtj4AP2LFA.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-TwCg28WTYcDvogNFQwGRHw.jar
May 12, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-2-lLi4x4EB1mUxXeRxRpgg.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-3FMXW65K56qN8FVYPpg4Ow.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-pasqcNv1dnqKen5xxRQmTg.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-w8IIKuxob-xja87-6EmrgQ.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-jJYPuyjvHRVLt0yzuV7Y-Q.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-K0x4aaioftWwIIIhOdaVUw.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-pebcTav3TPK6n7F9VHcMow.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-toEh82xjOm-SetL3r9iw2g.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-SrPBcNqG21XXC6AVetnJug.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-KE1DC8DoI5QhNcCgmlWELg.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-KE1DC8DoI5QhNcCgmlWELg.jar
May 12, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-XN6pHOykwPcedrl8VFObLQ.jar
May 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-KE1DC8DoI5QhNcCgmlWELg.jar&uploadType=resumable&upload_id=AAANsUk3Hd88-GisPxYr1tEFJR0qa8O-FuqPLuxdRuiBUtF8Ho2cS684D7c7XnXbFaVuxtg85O0dVGiz-mSPcUGykQhT93pyfg.
May 12, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 12, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-KE1DC8DoI5QhNcCgmlWELg.jar
May 12, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 12, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash rqm_FJzzP-3y_pHvMAXCHA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rqm_FJzzP-3y_pHvMAXCHA.pb
May 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-11_23_45_49-11932625991702942072?project=apache-beam-testing
May 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-11_23_45_49-11932625991702942072
May 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-11_23_45_49-11932625991702942072
May 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T06:45:49.048Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:55.380Z: Checking permissions granted to controller Service Account.
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.025Z: Worker configuration: n1-standard-1 in us-central1-a.
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.638Z: Expanding CoGroupByKey operations into optimizable parts.
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.679Z: Expanding GroupByKey operations into optimizable parts.
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.711Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.802Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.841Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.868Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:45:59.910Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:46:00.331Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:46:00.415Z: Starting 5 workers in us-central1-a...
May 12, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T06:46:24.200Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 12, 2020 6:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:46:28.249Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 12, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:46:45.750Z: Workers have started successfully.
May 12, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:46:45.778Z: Workers have started successfully.
May 12, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:47:21.056Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:47:21.275Z: Cleaning up.
May 12, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:47:21.374Z: Stopping worker pool...
May 12, 2020 6:49:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:49:16.047Z: Autoscaling: Resized worker pool from 5 to 0.
May 12, 2020 6:49:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T06:49:16.093Z: Worker pool stopped.
May 12, 2020 6:49:23 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-11_23_45_49-11932625991702942072 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3f53c91e-2c30-456e-a74f-e45d0ac6ddb0 and timestamp: 2020-05-12T06:49:23.543000000Z:
Metric: Value:
read_time 17.904
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 6:49:24 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 55.456 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/c4uhm4t5tr7ms
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #492
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/492/display/redirect?page=changes>
Changes:
[sjvanrossum] [BEAM-9776] Fixes filesystem not found error
[kcweaver] [BEAM-9875] Use FlinkRunner instead of PortableRunner in cross-language
[kcweaver] Remove runner check in xlang wordcount.
[kcweaver] [BEAM-9835] [Portable Spark] Broadcast a PCollection at most once.
[robertwb] [BEAM-9935] Respect allowed split points in Python.
[github] Add License file for the Go SDK for pkg.go.dev
[robertwb] More comments.
[rohde.samuel] Return empty iterator on empty cache
[github] [BEAM-9887] Throw IllegalArgumentException when building Row with
[klk] [BEAM-9952] Revert "remove PipelineRunner.fromOptions call"
[github] Merge pull request #11637 from Waiting for BQ Query and Export jobs for
[robertwb] Make grpc-requiring class definition optional.
[github] [BEAM-9884] Add options to SqlTransform for specifying planner (#11647)
------------------------------------------
[...truncated 294.83 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 1:04:42 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 1:04:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 12, 2020 1:04:42 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 12, 2020 1:04:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 12, 2020 1:04:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 12, 2020 1:04:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-zZcB56HOLUsy_1B7Y4owRg.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-rlQHDbrjeexzGNeLtVPSxw.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-CDf4tlJDqbUwJORyJ7yPVQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Ss-BZ6jLPq3GE6dxQwpG-A.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-QABkS0IMV7yqW3CUIHwUmg.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-EntwkbAiSD4gqsYONLsKsA.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-TlvUJvJJJ4oXG9biFoXMVw.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-3-eLhRD2P55r0Uit1w8FnQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-gRZvobDbpj9MwJYfoc8HNQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Q22laTHrNRaoIa1U0o4KAA.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-W5bGNIj9jd9epuDRhwqLHQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ap5MutDyCyh1Y69CdUNDrQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-EmvNzxpxj_Oy8blKpUSjSg.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-q-Eaan6ICHsc__hK8MOZig.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-82b9nuwSwas2gVpJxBGeJg.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-S34LiM_vCJpiQQmtfAVP8w.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-SxXk56ULCUfxdi9kRrBJDQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-JlP2qjKr-OaS43kGatzSxQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-lXNam55HI9qHczfNBG1HyQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WvawgNvRBjcfESOEAdDaJA.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-p2Rppa2izO81_Px5fPcQAQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WvawgNvRBjcfESOEAdDaJA.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-4qbgZG3SztyUeJNEt0hCUw.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-nywAshn6tmF3iCxdDYsEjA.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-MPDg1VbFmIjzKyyhb0lqQQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-P-_PFoWwy_kf1GrlZ3N0MQ.jar
May 12, 2020 1:04:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-P6s1N_3ZxPnbxFNFKKGboA.jar
May 12, 2020 1:04:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-5J4aP8EzJAsleAGPtEXQeQ.jar
May 12, 2020 1:04:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-DloVTg9ryXVty8nFCl_paA.jar
May 12, 2020 1:04:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-yGD0IYQJWuu9Bu4OjM8Q2A.jar
May 12, 2020 1:04:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-c6ACKbsP74AeIWSA01UgUw.jar
May 12, 2020 1:04:46 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WvawgNvRBjcfESOEAdDaJA.jar&uploadType=resumable&upload_id=AAANsUlSvtJAMX38b5FEhH8jJnwzLUlh2prhKCec6B7yErZNsI51zBlScDbM0ER3JnfgIGCSI_a0kEp9IXeep4xguYEz8iWmcQ.
May 12, 2020 1:04:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 12, 2020 1:04:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-WvawgNvRBjcfESOEAdDaJA.jar
May 12, 2020 1:04:51 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 12, 2020 1:04:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash ZaeLtFpbbcyprzeDTAX55Q> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZaeLtFpbbcyprzeDTAX55Q.pb
May 12, 2020 1:04:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 12, 2020 1:04:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-11_18_04_52-14362874060535167358?project=apache-beam-testing
May 12, 2020 1:04:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-11_18_04_52-14362874060535167358
May 12, 2020 1:04:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-11_18_04_52-14362874060535167358
May 12, 2020 1:04:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T01:04:52.828Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 12, 2020 1:04:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:04:56.040Z: Checking permissions granted to controller Service Account.
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:00.543Z: Worker configuration: n1-standard-1 in us-central1-a.
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.142Z: Expanding CoGroupByKey operations into optimizable parts.
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.185Z: Expanding GroupByKey operations into optimizable parts.
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.221Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.323Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 12, 2020 1:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.351Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 12, 2020 1:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.393Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 12, 2020 1:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.431Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 12, 2020 1:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:01.900Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 1:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:02.086Z: Starting 5 workers in us-central1-a...
May 12, 2020 1:05:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-12T01:05:23.711Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 12, 2020 1:05:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:29.571Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 12, 2020 1:05:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:29.612Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 12, 2020 1:05:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:47.472Z: Workers have started successfully.
May 12, 2020 1:05:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:05:47.506Z: Workers have started successfully.
May 12, 2020 1:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:06:22.167Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 12, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:06:22.366Z: Cleaning up.
May 12, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:06:22.451Z: Stopping worker pool...
May 12, 2020 1:10:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:10:16.799Z: Autoscaling: Resized worker pool from 4 to 0.
May 12, 2020 1:10:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-12T01:10:16.843Z: Worker pool stopped.
May 12, 2020 1:10:24 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-11_18_04_52-14362874060535167358 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b3a1e627-9691-4276-97e2-defebf4771fe and timestamp: 2020-05-12T01:10:24.380000000Z:
Metric: Value:
read_time 17.329
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 12, 2020 1:10:24 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 5 mins 52.095 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 20s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/lxgc3whvzncbe
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #491
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/491/display/redirect?page=changes>
Changes:
[kamil.wasilewski] Fix InfluxDB credentials binding
[borzoo.esmailloo] [BEAM-9216] Upgrade katas gradle to version 5.2.1
------------------------------------------
[...truncated 302.73 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 7:15:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 7:15:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 11, 2020 7:15:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 7:15:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 11, 2020 7:15:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 11, 2020 7:15:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 11, 2020 7:15:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 11, 2020 7:15:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 11, 2020 7:15:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 11, 2020 7:15:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-etNFVSMaF6IAiRuQA001zQ.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-aVvJ2234a9EnPDsaa6Xy1w.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-JW-FsylI8G8cwq3lCSDZtg.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ijPvFqK_338RNnpyGlk--Q.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-n2_zLpLysdRmefobRtPqfQ.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-DDQbfnFi0ULSiPSpCwURxA.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iWtTWh_FcgEsDWSbIfMZ0A.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-y20EnAeyM5bfE0uwGHEijw.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-HdxcgQk1kP0T1amiojR3tA.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-AkKrfHEDqo74bpMMGrrWQg.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-xZf92Woq0gOaJWbl7_zblg.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iWtTWh_FcgEsDWSbIfMZ0A.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-80-XUYiBIiTsr4afAaOeew.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-jDo3WdYjQoRRXF8jBbfD9w.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-EDhETVmXcL0knc8AlD4fNw.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-XPHdqNDWrMN9zFaR6XPHwA.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Xkgof_4-K-Y5_QkU7IrGGg.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-oiNPdxir_Ba4gKAmhRNDEw.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT--wnn250irLITAn3RhjI11g.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-KEPYxQP6k_zTeW2G7CL51g.jar
May 11, 2020 7:15:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-cNnfHAxufGOA4eSAkXDv-w.jar
May 11, 2020 7:15:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-hFoKafSdzK64rwUPzFJY-Q.jar
May 11, 2020 7:15:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-tKQhgSKUMIViGektMWbGMw.jar
May 11, 2020 7:15:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-tr54YQbRp3_Qe4MVabJH0g.jar
May 11, 2020 7:15:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-wp7y00SgzDgbIQ76b7hK9A.jar
May 11, 2020 7:15:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-7xT3C6i0GadtEkeZHb9fEg.jar
May 11, 2020 7:15:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-nLyBTJ6Li9SvWAgcMLzfNw.jar
May 11, 2020 7:15:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-uvRDQwvtlTSPgSW9U7f2aw.jar
May 11, 2020 7:15:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iWtTWh_FcgEsDWSbIfMZ0A.jar&uploadType=resumable&upload_id=AAANsUk62Mu1NeBG5zvqUbRVvpG_N5TxUT_1zQT1PMn_naDUprd6ahvb-b0GHoXozz924jdms2qvuViYMOYT-UDuIjRWxWKmNg.
May 11, 2020 7:15:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 11, 2020 7:15:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-RZF3wkBX3iTE3JYEqVHfew.jar
May 11, 2020 7:15:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Owmk6QqhqEd8seSM2bJshQ.jar
May 11, 2020 7:15:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-iWtTWh_FcgEsDWSbIfMZ0A.jar
May 11, 2020 7:15:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 30 files newly uploaded in 13 seconds
May 11, 2020 7:15:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 11, 2020 7:15:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 11, 2020 7:15:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 11, 2020 7:15:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 11, 2020 7:15:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 11, 2020 7:15:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash UCZNUJR7ifSJKxebMtdZLQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UCZNUJR7ifSJKxebMtdZLQ.pb
May 11, 2020 7:15:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 11, 2020 7:15:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-11_12_15_42-14565552513457172816?project=apache-beam-testing
May 11, 2020 7:15:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-11_12_15_42-14565552513457172816
May 11, 2020 7:15:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-11_12_15_42-14565552513457172816
May 11, 2020 7:15:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T19:15:42.426Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 11, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:53.402Z: Checking permissions granted to controller Service Account.
May 11, 2020 7:15:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:57.743Z: Worker configuration: n1-standard-1 in us-central1-f.
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.425Z: Expanding CoGroupByKey operations into optimizable parts.
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.466Z: Expanding GroupByKey operations into optimizable parts.
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.506Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.592Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.628Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.664Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:58.689Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:59.094Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 7:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:15:59.170Z: Starting 5 workers in us-central1-f...
May 11, 2020 7:16:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T19:16:19.996Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 11, 2020 7:16:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:16:28.208Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 11, 2020 7:16:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:16:44.551Z: Workers have started successfully.
May 11, 2020 7:16:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:16:44.590Z: Workers have started successfully.
May 11, 2020 7:17:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:17:20.448Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 7:17:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:17:20.715Z: Cleaning up.
May 11, 2020 7:17:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:17:20.785Z: Stopping worker pool...
May 11, 2020 7:19:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:19:23.882Z: Autoscaling: Resized worker pool from 5 to 0.
May 11, 2020 7:19:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T19:19:24.003Z: Worker pool stopped.
May 11, 2020 7:19:29 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-11_12_15_42-14565552513457172816 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b03d69e5-8c7a-4cb1-a80a-3b841a4ffa55 and timestamp: 2020-05-11T19:19:30.049000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.176
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 7:19:31 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.043 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.105 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4 mins 52.833 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 37s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ftlcdihv7oxq4
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #490
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/490/display/redirect?page=changes>
Changes:
[sebastian.graca] [BEAM-9439] Return split instead of total backlog size
------------------------------------------
[...truncated 292.05 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 11, 2020 12:48:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 11, 2020 12:48:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 11, 2020 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 11, 2020 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 11, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-t47bAl55H2hOshStYeFFpg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-rEhvALPiizneUzZfoPYt_Q.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-4YyVF8KZ20itvJe5yuz6Yg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-H6DFGXi4Z6_QPt51fVtrzg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-qiqMUnotl0saauj8RpX_lg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Y1VgtLv0JcgyNljQt09H2A.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ARLHWiANoDsV1svkXwErDA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-CcJJE0yI_yt6fyXxptCVTA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-U9SOf2_MBkaQhJHVmAZqYA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-9MpB3CodIIfRBa_sLsvCSg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-TXRt4i2Ue8PZGBjxTNHp4g.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-C8h2OzDddyzQM6s-pyfcZg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-jbmgFZkIfP6jBHkEqSPbKQ.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-bl6mHU_89A-prka_PVErEg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-PAaGm9k7O90DFHhVFzn4-Q.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ecalfcHpnuUb7fXciD8VGQ.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-kvt4GidHEpUK-COYZ9kp8w.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Ma91Mn3Rep57YPFxypJUZg.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ktkO3bUQrxzlU8o2p8q62g.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ce6I2DpDHD35OU88aqyWiA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-QKDhukzHKL9_C_7rlH0j8w.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-zkV_bVf8ajkferjvcJPkeA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-njuJiQW10TzRB8MXy1sMNA.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-m4opTG7iOKZHZpfolr0stw.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-_mT4zTbISSVlMdy6gNYRMQ.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-nX_-quvLzZMGrNK9TfYS2w.jar
May 11, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-H6vtvp_7j08UA78-mr4pxA.jar
May 11, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F9wZcu4BUCMZwA4OXqVXLQ.jar
May 11, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-dsrEQ_olCTdcI20yHjpJpA.jar
May 11, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F9wZcu4BUCMZwA4OXqVXLQ.jar
May 11, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ZPMCngjl1eJcv16KE-Uvdw.jar
May 11, 2020 12:48:24 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F9wZcu4BUCMZwA4OXqVXLQ.jar&uploadType=resumable&upload_id=AAANsUkcnLykWgUjFHJy36vQmTCWYsYRKtD-qUqiun1NGk0Gcgk6ziaZJo6vhz21_Pl-oQX49XF9e6tQMnbITwajMFM.
May 11, 2020 12:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 11, 2020 12:48:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F9wZcu4BUCMZwA4OXqVXLQ.jar
May 11, 2020 12:48:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 11, 2020 12:48:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash w6kJosqf3ZtrWQSuA5RAPw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-w6kJosqf3ZtrWQSuA5RAPw.pb
May 11, 2020 12:48:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 11, 2020 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-11_05_48_32-10326223276267831068?project=apache-beam-testing
May 11, 2020 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-11_05_48_32-10326223276267831068
May 11, 2020 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-11_05_48_32-10326223276267831068
May 11, 2020 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T12:48:32.141Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 11, 2020 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:35.024Z: Checking permissions granted to controller Service Account.
May 11, 2020 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:38.554Z: Worker configuration: n1-standard-1 in us-central1-a.
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.407Z: Expanding CoGroupByKey operations into optimizable parts.
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.454Z: Expanding GroupByKey operations into optimizable parts.
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.491Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.572Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.603Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.643Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:39.675Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:40.151Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:48:40.232Z: Starting 5 workers in us-central1-a...
May 11, 2020 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T12:48:53.531Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 11, 2020 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:04.469Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 11, 2020 12:49:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:23.839Z: Workers have started successfully.
May 11, 2020 12:49:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:23.875Z: Workers have started successfully.
May 11, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:59.308Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:59.502Z: Cleaning up.
May 11, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:49:59.604Z: Stopping worker pool...
May 11, 2020 12:52:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:52:42.928Z: Autoscaling: Resized worker pool from 5 to 0.
May 11, 2020 12:52:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T12:52:42.982Z: Worker pool stopped.
May 11, 2020 12:52:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-11_05_48_32-10326223276267831068 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 342f2d82-920e-4f9d-998c-8f10fe2d18fc and timestamp: 2020-05-11T12:52:50.218000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.953
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 12:52:50 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 4 mins 37.767 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 24s
103 actionable tasks: 67 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/ontk5xicjp3xs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #489
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/489/display/redirect>
Changes:
------------------------------------------
[...truncated 293.72 KB...]
May 11, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 11, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 11, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 11, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 11, 2020 6:45:29 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-SIJ74hiF-lRbV2GTFQFqQw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-7G0q0pSbTarJmJlelBBDtQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-PiPI2fbPI6rtzC3_SUSH3w.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ohu4Nv47tY6L8Zi4SbGDQQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-iWpDlsxWn9CCH0Adkn3ssw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ZIoedPRbkkpZd1sAJZMzvQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-VOyeEsilpn56QBEjcpTDZg.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-qbhMLQw8a3tgpivMqnjS1Q.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-XI4Unlk0nAE1CQRDlLpC6w.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Wn2KDLiy0zbuBQKLhmhc6w.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-NnZdVnijspyEU7Vdw-fu6Q.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-xV-2Nj2Twsfr7CNxHHVE-g.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-TdyCeZbUHZzVFCMhZZ6Cgg.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-S67_-GZzGAObGQeTopvT2w.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-P1fxuDjnRbVGOcnU0s3Qqg.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-Vl9x_LtckSp4ft9gefWEVw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-vzDwP8I24DMhLXheXIwovw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-aIPAi5ODxizer82ISZu0GQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-OytOAmq1xtc6XxN04_Ikjw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-IwoAIaZRgYi1it0nK5FZtw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-T5mavTQ0Q4wv9SPINcX0VQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-pcCayx9waX6aiVamLB2hnw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-YAPNw7ekvEiBD-YuC_ctAA.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-GG6EumLGjxW-pyHoiKSqRw.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FUSqyVY9CkWV6Xf7tG02gQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-uBuW2deJYZYzKjpX55-yHQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-nl-Del3Ue5XjAvwSz_Rj1Q.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-9o1o34DExO_EduPWRVSedQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FUSqyVY9CkWV6Xf7tG02gQ.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-mdlb3v5ayMw5aiP6tdxwAg.jar
May 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-WJA-j96fPCX2sCErEExPwg.jar
May 11, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FUSqyVY9CkWV6Xf7tG02gQ.jar&uploadType=resumable&upload_id=AAANsUnxurir25TYPQINNwX9-lAW2wPi6Mi7lu3EkkEoAnHuju7G0LEhTXhUi4Gjed-mnhmNeRLu2qWeP0RaSsQUzQ.
May 11, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 11, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FUSqyVY9CkWV6Xf7tG02gQ.jar
May 11, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 11, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash _OPQNcGvdzEKoYYx1Z2v4A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_OPQNcGvdzEKoYYx1Z2v4A.pb
May 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 11, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-10_23_45_41-2340606770947084064?project=apache-beam-testing
May 11, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-10_23_45_41-2340606770947084064
May 11, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-10_23_45_41-2340606770947084064
May 11, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T06:45:41.127Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 11, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:44.335Z: Checking permissions granted to controller Service Account.
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:47.304Z: Worker configuration: n1-standard-1 in us-central1-a.
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:47.991Z: Expanding CoGroupByKey operations into optimizable parts.
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.021Z: Expanding GroupByKey operations into optimizable parts.
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.050Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.123Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.179Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.202Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 11, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.224Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 11, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.513Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:45:48.571Z: Starting 5 workers in us-central1-a...
May 11, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:46:14.025Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 11, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:46:14.057Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 11, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:46:19.421Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 11, 2020 6:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T06:46:21.312Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 11, 2020 6:46:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:46:31.675Z: Workers have started successfully.
May 11, 2020 6:46:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:46:31.703Z: Workers have started successfully.
May 11, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:47:09.671Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:47:09.854Z: Cleaning up.
May 11, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:47:09.924Z: Stopping worker pool...
May 11, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:48:37.516Z: Autoscaling: Resized worker pool from 5 to 0.
May 11, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T06:48:37.552Z: Worker pool stopped.
May 11, 2020 6:48:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-10_23_45_41-2340606770947084064 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 99597c8e-d3b8-462a-bd53-c6c8aa60b7ac and timestamp: 2020-05-11T06:48:42.906000000Z:
Metric: Value:
read_time 19.447
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 6:48:43 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 21.912 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/ubezc4spthoge
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #488
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/488/display/redirect>
Changes:
------------------------------------------
[...truncated 293.55 KB...]
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 11, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 11, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 11, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 11, 2020 12:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 11, 2020 12:45:16 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-V7S-rCdVoSvYdsKMIQQTVw.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-0cdqt7ZUcKhsz1-BVYNA0w.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-crNdow3PE1OM9N0bb9SRQQ.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-fRDABseqM8up6zv-UMzjWg.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-NoKh_tHVBlpaSJnkreRZ0A.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-fbuRcMCHAixZp1VWBjm8ww.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-T-X_uvjbMAZK1CL9fA9OIQ.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-PkdyvtWq2-kR7UzY3XvttA.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-TXqOMO-hAxE17AxuCoJNlA.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-auTiKkF3BEMAs6r8NBvYFQ.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-d6tS7s3YiqwtD-3Bc16DDg.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-qgqFkMZWaT7QPVzdWC5-HA.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-oeQVB_inhJjfO9xxmibZHA.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-n037jyyDNZrmwxf4NeYHhA.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-rw81ujdQhqFleR7xJW_KSw.jar
May 11, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-QHvzV9MNPJ03vEv5u9yt6Q.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-_4FO80eEAyXI-KuicVtBQA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-JE86lPKMbVImfbBo3aqTsA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-baVx-cVPFSZP1DYhNv3ZIQ.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-pdWVWYd4XYLirsZTp-661A.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-xgGERmNGPIeoEXvtaAvqPg.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-kgSui11V4Oam7jM-A32HNQ.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-xfCcMckERxL-9jNa9HE7ew.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-b3aj7i7dqcCfQhIXohpzfA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-gJNJriFxxJwCSpzu3IW-aA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mockito/mockito-core/3.0.0/15fd0225cb1858f6922f44776b1577ac26739279/mockito-core-3.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/mockito-core-3.0.0-7tOnWjwKUa3vSvwalF3i1w.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-6LJw-ZTrE-C-P-gh4Im00w.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/byte-buddy-agent-1.9.10-BCkqVrRfNOFZg1RAm6NEQw.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4mnMYC81fwYnGvfAkkB5qg.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar to gs://temp-storage-for-perf-tests/loadtests/staging/byte-buddy-1.9.10-QusrhKJgPcRx1AnckpJw7Q.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vkfnhS3XJz5u4BNZyPdBRA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-afwmmpvQS7jzlTkfzyTPZw.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-pQjjLFhs6r-xkZ-lTRI_eA.jar
May 11, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vkfnhS3XJz5u4BNZyPdBRA.jar
May 11, 2020 12:45:19 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vkfnhS3XJz5u4BNZyPdBRA.jar&uploadType=resumable&upload_id=AAANsUn9uvAzgi04tjaE8AcrEvNk9eCEL-gwSmc_FShZGWbASx36izCsLsHSG5gSbVZ17nTIw5uYEafYuK6WrzIWd4fYthhGUg.
May 11, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 11, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vkfnhS3XJz5u4BNZyPdBRA.jar
May 11, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 176 files cached, 34 files newly uploaded in 8 seconds
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 11, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash lP8FoKabaLz8Im4cJXY2eQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lP8FoKabaLz8Im4cJXY2eQ.pb
May 11, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 11, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-10_17_45_27-5449013616921818184?project=apache-beam-testing
May 11, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-10_17_45_27-5449013616921818184
May 11, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-10_17_45_27-5449013616921818184
May 11, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T00:45:27.201Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 11, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:30.556Z: Checking permissions granted to controller Service Account.
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:34.460Z: Worker configuration: n1-standard-1 in us-central1-a.
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.128Z: Expanding CoGroupByKey operations into optimizable parts.
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.157Z: Expanding GroupByKey operations into optimizable parts.
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.177Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.244Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.268Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.292Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.317Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.595Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:45:35.656Z: Starting 5 workers in us-central1-a...
May 11, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-11T00:45:57.191Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 11, 2020 12:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:03.280Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 11, 2020 12:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:03.302Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 11, 2020 12:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:08.913Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 11, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:25.592Z: Workers have started successfully.
May 11, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:25.613Z: Workers have started successfully.
May 11, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:46:59.843Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 11, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:47:00.040Z: Cleaning up.
May 11, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:47:00.122Z: Stopping worker pool...
May 11, 2020 12:48:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:48:28.983Z: Autoscaling: Resized worker pool from 5 to 0.
May 11, 2020 12:48:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-11T00:48:29.023Z: Worker pool stopped.
May 11, 2020 12:48:35 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-10_17_45_27-5449013616921818184 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 776098fc-7e67-415e-a4b3-7666e4d3a7c7 and timestamp: 2020-05-11T00:48:35.229000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.286
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 11, 2020 12:48:35 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 27.601 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 17s
103 actionable tasks: 67 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/o6wma3ouyoyf6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #487
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/487/display/redirect>
Changes:
------------------------------------------
[...truncated 295.43 KB...]
May 10, 2020 6:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 10, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 10, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 10, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 10, 2020 6:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2020 6:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-sYrmAIktmUJxUNl80553SA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-TC6KjpxNM9gyObpu-TJutg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-D8wN1YAD39k0ibYxkNYoxg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-THrwzi9_XNW2yAJCYXsKQg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-hHT0aprDwR0N9pQsphVq1A.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-YWr6nZG-chb_j_MGecumHg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-pvTquNVsNxnHfHMijQ5SOA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-U8-VV87hDOx-EwrhhtqxQA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-z_VUaEQrb2V8qEoClKZ6iQ.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-0qRUdasoSG0vDc6Op3BDKQ.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Zh4ZMjSxDDo7iQ78h0z-wA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-gd5R0J5dJzH4PC5GhVK8Qg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-M6RzN6ORwFohd9l7Sj7HiA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-X8Yb_FrfO5HYExLNwZ_ggA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-L482nB54bMWQBzGG1Eu71g.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-XVQlRFar-kPOXm2_IDhAyQ.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-mPsJXgxFR0VaZMAoQ_tKyA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-aDZ97PDsQzMIlAnHUymZpQ.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-HkCqT7blaA8Rec-QLNBqkQ.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-XUXGUY2u5sF877C6MmQdUg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-sTRWnfsDiCVPZsmIiQ032A.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-ZTRZ-PD-koJjg9Sz7i32lg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-sGnSrVKLemRATp3O4sXhRg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-WPqqMTvpv7kYlofArwr3JA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-XT83Vxd2PiBGa1JnEG2LJg.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-HBmHfuHWYKR0h7zYuAGuSA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ek_O7BbImCo-1uGRuR2Qrw.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JWQ_Uc1Kq2gPbI_5rjREKw.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-CImvKpPhvv6rU0KtSjQauw.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-XEY2hHTSsfHgf4T-82ImtA.jar
May 10, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ek_O7BbImCo-1uGRuR2Qrw.jar
May 10, 2020 6:45:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ek_O7BbImCo-1uGRuR2Qrw.jar&uploadType=resumable&upload_id=AAANsUmvQ5E_WMHVGUthvSjxFQZJrEbX4jRm_nFVL-Fs7XWhtDFDPHsB7G2Sf5AHbTV36hlmpbqzzqqOZ9FcECW_zw4.
May 10, 2020 6:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 10, 2020 6:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ek_O7BbImCo-1uGRuR2Qrw.jar
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash SZy9zdP-8yXNn_o6OK3lYQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SZy9zdP-8yXNn_o6OK3lYQ.pb
May 10, 2020 6:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 10, 2020 6:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-10_11_45_33-531396056403261245?project=apache-beam-testing
May 10, 2020 6:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-10_11_45_33-531396056403261245
May 10, 2020 6:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-10_11_45_33-531396056403261245
May 10, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T18:45:33.448Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 10, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:36.676Z: Checking permissions granted to controller Service Account.
May 10, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:41.327Z: Worker configuration: n1-standard-1 in us-central1-f.
May 10, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:41.933Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:41.976Z: Expanding GroupByKey operations into optimizable parts.
May 10, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.015Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.104Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.134Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.171Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.210Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.554Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:45:42.642Z: Starting 5 workers in us-central1-f...
May 10, 2020 6:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:46:14.268Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 10, 2020 6:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:46:14.308Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 10, 2020 6:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T18:46:16.029Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2020 6:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:46:19.761Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 10, 2020 6:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:46:34.025Z: Workers have started successfully.
May 10, 2020 6:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:46:34.057Z: Workers have started successfully.
May 10, 2020 6:47:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:47:06.729Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 6:47:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:47:06.922Z: Cleaning up.
May 10, 2020 6:47:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:47:07.013Z: Stopping worker pool...
May 10, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:48:42.284Z: Autoscaling: Resized worker pool from 5 to 0.
May 10, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T18:48:42.333Z: Worker pool stopped.
May 10, 2020 6:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-10_11_45_33-531396056403261245 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b904e9e4-812d-4719-bc0f-9f80a253feef and timestamp: 2020-05-10T18:48:48.111000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.26
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 6:48:48 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 33.418 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 31s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/sqqaub2l7wuqu
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #486
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/486/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9930] Beam Summit Digital 2020 announcement on blog
[mxm] Correct URL in blog post
[matthias.baetens] Minor (markup) changes
[mxm] [BEAM-9930] Correct authors of beam summit digital blog post
------------------------------------------
[...truncated 289.99 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 10, 2020 12:45:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 10, 2020 12:45:12 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2020 12:45:12 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-jEzPN7y2RzXI1LtE56hQ6w.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-0S1OCJ9shfKS5TfpISOl3Q.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-QuOQ-3m5YaQgXLMSf1RFiA.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-pCZQad_w9ra_KVjO9D6dtg.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-XrWVT3ZK0-jDThvMQF7AcA.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Y9MbVUxs3dj2w-0Ed2lh-w.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-iDgusGGY9--BdJU63gi5WQ.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-3nrv9gGS_CndF8UOcQ6FBA.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Gij-XhlQBaSsdAMwPkg6-Q.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-SJ06i0M-eJT4hI57MAqmAw.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-NgyEswFbWPFPdbhnY3mjbw.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-1xiJsoM4oPF-oFrI0nQcRQ.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-mv26YWSbi3mRLdjWrgsl6g.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-dwcyVwmZzTCro9SkEB1Qmg.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-ixhKlifD3TTQAuFRGWH5ng.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-BHzDeWDr1YQPDS7O-XUjwA.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-uta8JtO5h81vfM42bKev-g.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-6CzeqVB7c1ITvChSFWix-Q.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-gh-bExV1pOplS96D8YMSKQ.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-nAF32pjF9J6E2ReekLkZDw.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-DZcXKOe3ASgyd-aZ5WZJzA.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-0nPHy8MKM3mYn3xgalZO-w.jar
May 10, 2020 12:45:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-I5zPbyFD7pc6xzICQ8iLxQ.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-k7RAZ0Xxg5kHOG4spHqDWA.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-u50sdca2t_fccU3l-Xn5Eg.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-fmRWTmTAiYZUhY8Y3PL_Ww.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-aTFyXVnmVHU9z7YQ2NJ6pg.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-XHqU6iKrsx7L69n7tumaSQ.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-81-MTCeGcQiLhAwDEKr-Cw.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-rPv6tQwlqK1wdpnrWeIkzg.jar
May 10, 2020 12:45:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-81-MTCeGcQiLhAwDEKr-Cw.jar
May 10, 2020 12:45:15 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-81-MTCeGcQiLhAwDEKr-Cw.jar&uploadType=resumable&upload_id=AAANsUnXJvPwsnweV4uEA-aVIGZy-AkWtKdv8TQ3Zil4Rd4kyo1Nkl2fog5LDDRLSqHfkCQyFyGUtw2fC8gWeh2_9g.
May 10, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 10, 2020 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-81-MTCeGcQiLhAwDEKr-Cw.jar
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 6 seconds
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 10, 2020 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2020 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12103 bytes, hash N-S1gePS0me_WJ56c4X7aw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-N-S1gePS0me_WJ56c4X7aw.pb
May 10, 2020 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 10, 2020 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-10_05_45_20-12503475785597873630?project=apache-beam-testing
May 10, 2020 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-10_05_45_20-12503475785597873630
May 10, 2020 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-10_05_45_20-12503475785597873630
May 10, 2020 12:45:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T12:45:20.576Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 10, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:23.954Z: Checking permissions granted to controller Service Account.
May 10, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.017Z: Worker configuration: n1-standard-1 in us-central1-a.
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.680Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.749Z: Expanding GroupByKey operations into optimizable parts.
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.779Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.838Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.859Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.883Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:28.911Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:29.689Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:45:29.747Z: Starting 5 workers in us-central1-a...
May 10, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T12:45:55.372Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2020 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:00.203Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 10, 2020 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:17.631Z: Workers have started successfully.
May 10, 2020 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:17.663Z: Workers have started successfully.
May 10, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:55.398Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:55.607Z: Cleaning up.
May 10, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:46:55.678Z: Stopping worker pool...
May 10, 2020 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:48:41.423Z: Autoscaling: Resized worker pool from 5 to 0.
May 10, 2020 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T12:48:41.470Z: Worker pool stopped.
May 10, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-10_05_45_20-12503475785597873630 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0ac6a01b-d81e-4791-a1a3-347f2866d056 and timestamp: 2020-05-10T12:48:49.238000000Z:
Metric: Value:
read_time 20.782
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 12:48:49 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 45.159 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
103 actionable tasks: 67 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/mvlzwxmshjj7e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #485
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/485/display/redirect>
Changes:
------------------------------------------
[...truncated 293.83 KB...]
May 10, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 10, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 10, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 10, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 10, 2020 6:45:27 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Qbv2Ep44kjOMXv20xFYJKA.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-LUBIWtkhQP27Rv9P8iWusw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-g6ABmOXU-OS_mEODv9YtLA.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-E4m2y6vfrww8L0DKMXydzw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-nSbKyNwtV16mcHfqmYWfdA.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-h7iTQCFH9GpGnT568qKY9Q.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-lFxkNnD6HK2GZvV1cuZjyw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-FcosfLT8zdrV1SlcFkpkOQ.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-YMkEJ8-dqB_NnhAKJ3Ndsg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-MnAHERnPDxzUDBZSCy7eAw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-vTitd0GJzCMOHoT2qdcpUw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-t3XS60wt1G77n6tdCZS5yQ.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-6_L8wHbF8jNJGVVJCEtBrg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-7yk4W3DIpqKtL5bZ79IJGw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-gqDY5hYz0qzW06ZXPsGmRg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-WxysGiZVZJ3pUQVUcjJpJw.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-6Tl7CsMhl7hVRUFpAqzdIA.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-a3Aw2arbqeXRWpeAcSapeg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-wsIFwD1x3beQ3tU_0TVdRg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests--lphw8KRNMMYCNxocDAT7w.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-UMiyvw-0rsuooKTRMtCWfg.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-vMSwOwXdGwrrUzcdvr-8JQ.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-oZQ5oWPr__HyLIHVuq445A.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-csv-1.8-kBnZnWBy9I_ZsfkJ8MRdJA.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-HZxjMC451mCwjHr9ITPs0Q.jar
May 10, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-TZkobVsGUILjpnUsLIRo_A.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-TSZT16wAtgoPSHhwV_HgLw.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-LuEQ_izgA0sf4ZcvhUE9xA.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LsMtELMH8IYUe3pJ-QUjZg.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-Vx2yjY9_UQUF398EUF7tSA.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-i4pErCFoT1CverDpIWpdrA.jar
May 10, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LsMtELMH8IYUe3pJ-QUjZg.jar
May 10, 2020 6:45:30 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LsMtELMH8IYUe3pJ-QUjZg.jar&uploadType=resumable&upload_id=AAANsUl3Crsyo0ghdOxpT5Q55aTO626-71Lr_j7F1TrtTj1-yQcGOo69OCSiLVdq4LfaFltTqpgwJbEacfdc9s8zWk67HfenkA.
May 10, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 10, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LsMtELMH8IYUe3pJ-QUjZg.jar
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 32 files newly uploaded in 7 seconds
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 10, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash vAjD8VfhirvcHsJRIMzS5w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vAjD8VfhirvcHsJRIMzS5w.pb
May 10, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-09_23_45_36-13329687656543510065?project=apache-beam-testing
May 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-09_23_45_36-13329687656543510065
May 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-09_23_45_36-13329687656543510065
May 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T06:45:36.738Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:45.608Z: Checking permissions granted to controller Service Account.
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:49.888Z: Worker configuration: n1-standard-1 in us-central1-a.
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.585Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.625Z: Expanding GroupByKey operations into optimizable parts.
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.667Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.765Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.797Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.835Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:50.870Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:51.245Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:45:51.324Z: Starting 5 workers in us-central1-a...
May 10, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T06:46:16.907Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:46:17.941Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 10, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:46:17.983Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 10, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:46:23.375Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 10, 2020 6:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:46:36.024Z: Workers have started successfully.
May 10, 2020 6:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:46:36.060Z: Workers have started successfully.
May 10, 2020 6:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:47:06.487Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 6:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:47:06.798Z: Cleaning up.
May 10, 2020 6:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:47:06.898Z: Stopping worker pool...
May 10, 2020 6:48:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:48:43.784Z: Autoscaling: Resized worker pool from 5 to 0.
May 10, 2020 6:48:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T06:48:43.849Z: Worker pool stopped.
May 10, 2020 6:48:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-09_23_45_36-13329687656543510065 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d7cfbe6b-05f8-4ac6-8bff-90e98f07c42d and timestamp: 2020-05-10T06:48:49.894000000Z:
Metric: Value:
read_time 12.774
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 6:48:50 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 31.008 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 32s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/kfminvmj7kzvy
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #484
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/484/display/redirect>
Changes:
------------------------------------------
[...truncated 292.69 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 10, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 10, 2020 12:45:30 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-hWjZVfnXjTCMcfxdgVCxyA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-uQ1OeOwjO06kbdKuRSe7Bg.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT--9AT5d-aV0A3g_xDkS0IXA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-gIrQYFPTDhsxttXBp4C5Bg.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-yStW9Hfn7Oqgnuixt-KUWQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-t58kb9fpzDRN6XArjUwhrQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-903QPfehUiddgZUP2a80bw.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-U_hn7haBQQGcbBOQjfjQWg.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-TrW6wknCuRZ-UV4ehg_2yQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-eKMM8W5c4eIBvXUdJevrPw.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-7HWiLhtXtmkjKY22Dr4JEQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-bmDgttgOAWxzVBcFE-4gIA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-omjYdaczZnjrEZTGp8FDlQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-FSjEvJ0YgZlOvCHm2c8cxA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-k-wwvtmvqeXDz-Ba00XQ5w.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-wEvedyDPZ67REYlYgH9sIQ.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-j28R5X4chYRmuXg7r8dkaw.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-7EBTTh1DciKkvxupBAg1DA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-PthgkPHiMn8vmzqQP3qZ4g.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-BEL6Xz9H_AeqsBrvZCLI5g.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-zu34QycD3AjjiVD0zXJKrA.jar
May 10, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-F8kwboHgTWERXVtvbiqkwg.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-vbz-dySc16H9cPjCoacxDg.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-S08_oTtJKSOvT4YX11wLPQ.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-JjmwIUJ1bwvpkbdCaPr8FA.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Bv6ckTe-g2Mlv2SsPzRqgA.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-FtS-pOhAMqMRBzd2S7fisw.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kaFy7kwz141ZIJPnDz987A.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-slLx9yt7zSQeklalRIvH2w.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kaFy7kwz141ZIJPnDz987A.jar
May 10, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-YR_XOpSgYa2rAy5lUl5Yew.jar
May 10, 2020 12:45:33 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kaFy7kwz141ZIJPnDz987A.jar&uploadType=resumable&upload_id=AAANsUnJsquH-OaFLY8DZyiLZ6SzaDGnPuNJzOefyDTmieJUkujXR8px1aMNqwYCGKCKPXQf7GUwwVpzQKMVe55tTCw.
May 10, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 10, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kaFy7kwz141ZIJPnDz987A.jar
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 10, 2020 12:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash n0w9Reo15FNippU6aDTFtA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-n0w9Reo15FNippU6aDTFtA.pb
May 10, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 10, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-09_17_45_39-5399519172386796920?project=apache-beam-testing
May 10, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-09_17_45_39-5399519172386796920
May 10, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-09_17_45_39-5399519172386796920
May 10, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T00:45:39.608Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 10, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:42.656Z: Checking permissions granted to controller Service Account.
May 10, 2020 12:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:46.721Z: Worker configuration: n1-standard-1 in us-central1-a.
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.399Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.444Z: Expanding GroupByKey operations into optimizable parts.
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.489Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.583Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.624Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.655Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:47.691Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:48.229Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 12:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:45:48.313Z: Starting 5 workers in us-central1-a...
May 10, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:46:20.480Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 10, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-10T00:46:22.670Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:46:35.767Z: Workers have started successfully.
May 10, 2020 12:46:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:46:35.807Z: Workers have started successfully.
May 10, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:47:09.736Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 10, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:47:09.933Z: Cleaning up.
May 10, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:47:10.022Z: Stopping worker pool...
May 10, 2020 12:48:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:48:52.022Z: Autoscaling: Resized worker pool from 5 to 0.
May 10, 2020 12:48:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-10T00:48:52.069Z: Worker pool stopped.
May 10, 2020 12:49:01 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-09_17_45_39-5399519172386796920 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): fc4e9178-92c5-4275-8be9-c529bdb21a7f and timestamp: 2020-05-10T00:49:02.009000000Z:
Metric: Value:
read_time 14.957
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 10, 2020 12:49:02 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 40.148 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 44s
103 actionable tasks: 70 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/gv7cdecpvhb7m
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #483
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/483/display/redirect?page=changes>
Changes:
[chambers] Add a BeamJava test with Flatten with different input and output Coders
[jferriero] check lastAttemptedOffset non-null
------------------------------------------
[...truncated 293.33 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 09, 2020 6:54:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 09, 2020 6:54:52 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2020 6:54:52 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-8Ug7Tz8yCo4BAlHybSQ_vw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-9JnIXEZDBsFnygadA9W5zg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-qh59n65kXJkczSXnWP_hBw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-FiJ7rPO2UOZTEhfjL0h5lw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-cJ6R6kZ-Kia0EJz1Ge2Uyg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-j8p5QXBHyH-JJfbnCDbO4Q.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MNYBuB2WZhktRzMajCF5hw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-CjALaNWkTlryHrNdx0QsUA.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-qtJAVqraSTgS5unsdjLdxg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-hwmBzlkFB2Jlsd9_AW7nhQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-yie2Ji_SNWd22Y4JLvLcOA.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-xQlWSCfcu2DmnQT0eBorGQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-4bg01TV3QKy2HA2QKteE_w.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-p_9AcX9HXsQFtNQLc2W2qQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-XqkmO1WEp8xM3aaJKbKvEw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-3_i7sGc6wKzepRvmu504OQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-mATXvZl22QWGffLHigIPKg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-16mS0iHUNfnMN-LTmzs6cg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-GYrn9IGoMBQAddsinI33hA.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-4eJ-lVoLtnUpL4QRVFu15Q.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-snJCskeMGOVQcCO2UDFdrw.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-Rvy72Lj17_DJO2J0aB12nQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-WSuvJjF4pMAsDxB7LtfjNg.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-5B8pZtc535TWAJLKiYif5g.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3MaDnWWnSiilQVp3snKjhQ.jar
May 09, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-2hNbnuQfKHH_1jrrzcEuhQ.jar
May 09, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-sDuswIIjHVcRVCNTify-Hg.jar
May 09, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-6yeNKoOf8J5t9ex3-AC-lw.jar
May 09, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-e-NN1lChRQ8f1IuxWJpM9g.jar
May 09, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-e-NN1lChRQ8f1IuxWJpM9g.jar
May 09, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Wvq6c-kkHdD2MzURH0OFkw.jar
May 09, 2020 6:54:55 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-e-NN1lChRQ8f1IuxWJpM9g.jar&uploadType=resumable&upload_id=AAANsUnpauQAEZqiSNpCt0QaN0KpaYkZNzWwhzGO1cK07wgezUPHyAzLdPgMahQhI0YkDfZ5-56ZlEwu0gTpdbckYw.
May 09, 2020 6:54:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 09, 2020 6:55:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-e-NN1lChRQ8f1IuxWJpM9g.jar
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 09, 2020 6:55:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2020 6:55:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash j5p0E4PtEN-HysEUP3r1UA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-j5p0E4PtEN-HysEUP3r1UA.pb
May 09, 2020 6:55:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 09, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-09_11_55_03-9292360769642694410?project=apache-beam-testing
May 09, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-09_11_55_03-9292360769642694410
May 09, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-09_11_55_03-9292360769642694410
May 09, 2020 6:55:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T18:55:03.853Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 09, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:06.922Z: Checking permissions granted to controller Service Account.
May 09, 2020 6:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:11.269Z: Worker configuration: n1-standard-1 in us-central1-a.
May 09, 2020 6:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:11.939Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2020 6:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.052Z: Expanding GroupByKey operations into optimizable parts.
May 09, 2020 6:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.082Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.155Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.191Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.230Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.261Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.618Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 6:55:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:12.696Z: Starting 5 workers in us-central1-a...
May 09, 2020 6:55:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:41.089Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 09, 2020 6:55:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T18:55:41.715Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2020 6:56:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:58.314Z: Workers have started successfully.
May 09, 2020 6:56:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:55:58.357Z: Workers have started successfully.
May 09, 2020 6:56:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:56:32.952Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 6:56:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:56:33.166Z: Cleaning up.
May 09, 2020 6:56:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:56:33.243Z: Stopping worker pool...
May 09, 2020 6:58:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:58:23.795Z: Autoscaling: Resized worker pool from 5 to 0.
May 09, 2020 6:58:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T18:58:23.827Z: Worker pool stopped.
May 09, 2020 6:58:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-09_11_55_03-9292360769642694410 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 515548d8-4028-4abc-8f99-25f79ff0502d and timestamp: 2020-05-09T18:58:30.123000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.551
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 6:58:30 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 46.689 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/sjstr2knhmyjg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #482
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/482/display/redirect>
Changes:
------------------------------------------
[...truncated 294.46 KB...]
May 09, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 09, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 09, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 09, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-H2cQ_oLKMcPLJwpbZyqXnA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-0Jc7uXlDTplWnBvEYXyyBw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-r0a0SPsV3c-p2R4UV8dOrA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-sOggULTbqYNAABHZhm3ioQ.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-_XE09z916Yr-z_koTGGC1w.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-QG7r5XhPRjHvWB180GDOKw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-hVDjgQugB-FQX4l0h-ubYg.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Z_U7XYBk2tYcZpF6u-Pzrw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-nJk0VvQx7GqAsH28DvhWAw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-PsIFqzL_oQ4_YRI0mQ-Q_w.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ktFwmPOU9hjFNSR8ACfPyg.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-w3zofYzwIKypK-x80Vxc8g.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-p4v-gYnggmIo0cmBBf2ghQ.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-g2ekZHDXUyIEiOJpvxVcZQ.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-jUY5ZHrs4PeKXXhGOfRzzA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-5EajC4RjFCiXJRjW9bq1Zw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-HacAMXK3BORN68NrPLmqfA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-jJ9akgAvpzSxG8QcoGvptQ.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-4o3pfFFspxcCHYVwp5pjYA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TElK6i2fYAhXUCBw7urCZQ.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-WbAJi0-cT7FqBiM9uZ6MpA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-2dL9v1RiizPB3HeGusW6mA.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-vsU4WzT7JCpKFibDtIANtw.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-GHQpgTPFRwZCba3ixsnyYg.jar
May 09, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-6Xai4p5byEkI_rg2SUaf4A.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-TlZmbqNGddRjtLWr5HMYJg.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Nc0tztM4jKfMLlIaDmTLbQ.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8n1ihgswpXcOW3M4KpgaAg.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-WrbxOcXEBOc4qoXZxF9pfA.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Vzlcnf_gprSX5n_eBKnH5Q.jar
May 09, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8n1ihgswpXcOW3M4KpgaAg.jar
May 09, 2020 12:45:42 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8n1ihgswpXcOW3M4KpgaAg.jar&uploadType=resumable&upload_id=AAANsUmMk_LsOCruRhNdbgeyIFiOj6nJfjZ-a50qvGcVnqX3QDelSvY1NmDQEY4cS2Z6LHwVURhUC2M175c0IUt8MTE.
May 09, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8n1ihgswpXcOW3M4KpgaAg.jar
May 09, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash fZY9PODy1QeaSk_GcAWdZA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fZY9PODy1QeaSk_GcAWdZA.pb
May 09, 2020 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 09, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-09_05_45_48-12763686506148008624?project=apache-beam-testing
May 09, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-09_05_45_48-12763686506148008624
May 09, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-09_05_45_48-12763686506148008624
May 09, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T12:45:48.254Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 09, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:51.547Z: Checking permissions granted to controller Service Account.
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:56.315Z: Worker configuration: n1-standard-1 in us-central1-a.
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:56.993Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.172Z: Expanding GroupByKey operations into optimizable parts.
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.212Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.294Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.318Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.349Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.384Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.757Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:45:57.842Z: Starting 5 workers in us-central1-a...
May 09, 2020 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T12:46:02.836Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2020 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:46:22.499Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 09, 2020 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:46:22.531Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 09, 2020 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:46:33.257Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 09, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:46:51.135Z: Workers have started successfully.
May 09, 2020 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:46:51.172Z: Workers have started successfully.
May 09, 2020 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:47:25.545Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:47:25.734Z: Cleaning up.
May 09, 2020 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:47:25.830Z: Stopping worker pool...
May 09, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:49:10.030Z: Autoscaling: Resized worker pool from 5 to 0.
May 09, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T12:49:10.076Z: Worker pool stopped.
May 09, 2020 12:49:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-09_05_45_48-12763686506148008624 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7b6eab9f-b4e8-446e-a7df-2c00f205634c and timestamp: 2020-05-09T12:49:15.445000000Z:
Metric: Value:
read_time 17.347
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 12:49:15 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 45.911 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/si52gjlybnx2w
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #481
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/481/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-9940] Set timer family spec for TimerDeclarations in dataflow
------------------------------------------
[...truncated 292.18 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 09, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 09, 2020 6:45:15 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2020 6:45:15 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-j3tOBg7klhjxhIexp963cg.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-5_EBZSHdEMUjPhCjHZq4gw.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-OCJ-vBPlrWpZ0CLnGv6HcQ.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-ZhhgWy-9FbQIATjobN1m9g.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-AvfUUB4K5al8tbpaUixHiA.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-A1MSqxPzUoe8HDQE0zudCg.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-6B3NwrPLLM8dRs8KqucPUQ.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-VO3M0m1DbdsFp7ny7cywDg.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-rx-8Jw3R9IEtdX7kd6EZ4w.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Ifca43peQA4Qz01BthV9iA.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-_mfnIt21gCCjTSQKk4Qh9Q.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-6xKxuHif_msAeKu-3BLbgw.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-xTLt1q7WIAmpr21QfbLk7w.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-4G6ji7_4CYHbpgwGt-RxSQ.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-l-WV1tmy4Cm3WVdfDXzmgg.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-zlxW807X8d8M2Toh5xcw5g.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-AQMKq0V7RFC4V_vxGFcShA.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-r5unZqa-BI_OcPxlp7vkAQ.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-bDCfF58B9W8o-iQvMANrtQ.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-xnO6mUzMenkII8xG4uA-Hw.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-dEteJ-HN85GUznn30OO7yA.jar
May 09, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-XphjzgWCPhUncJ4TftVCSw.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-j5hhaR97XDFAIKGc0CWRnA.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-XlB-De863UWFVLfzZR49hw.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-tR_38a6IA1aIewstsdL-Sg.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-pt3Flh6NH8GKBU63jGNfVQ.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8Pv99HK14DHZSCIqmUln1w.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-o-p1R0UsIWvRxKo-2rgldg.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tdlUvHiO_4N0dzYd6fIQkw.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8Pv99HK14DHZSCIqmUln1w.jar
May 09, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-Ds_-IzU4FaBikgvlybU_WA.jar
May 09, 2020 6:45:18 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8Pv99HK14DHZSCIqmUln1w.jar&uploadType=resumable&upload_id=AAANsUkgdR8b-giCDkFwXOEVhSAs6NoRr2uer3H4Eu_gyl7pvPIiBnchFxji3vKlRJg-MpKgf2qejx5UQHK5ImjM4Lki7d2Fvw.
May 09, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 09, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8Pv99HK14DHZSCIqmUln1w.jar
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 9 seconds
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 09, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12098 bytes, hash Zyys2kmN8UnFChukObajDw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Zyys2kmN8UnFChukObajDw.pb
May 09, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 09, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-08_23_45_26-17807443051829925135?project=apache-beam-testing
May 09, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-08_23_45_26-17807443051829925135
May 09, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-08_23_45_26-17807443051829925135
May 09, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T06:45:26.685Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 09, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:29.895Z: Checking permissions granted to controller Service Account.
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.102Z: Worker configuration: n1-standard-1 in us-central1-a.
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.769Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.807Z: Expanding GroupByKey operations into optimizable parts.
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.840Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.910Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.947Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:35.981Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:36.019Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:36.423Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:45:36.499Z: Starting 5 workers in us-central1-a...
May 09, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T06:45:58.105Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:02.941Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 09, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:19.396Z: Workers have started successfully.
May 09, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:19.434Z: Workers have started successfully.
May 09, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:56.028Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:56.250Z: Cleaning up.
May 09, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:46:56.350Z: Stopping worker pool...
May 09, 2020 6:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:48:42.960Z: Autoscaling: Resized worker pool from 5 to 0.
May 09, 2020 6:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T06:48:43.000Z: Worker pool stopped.
May 09, 2020 6:48:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-08_23_45_26-17807443051829925135 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3a81f301-93dd-4c58-9314-82163051e284 and timestamp: 2020-05-09T06:48:49.820000000Z:
Metric: Value:
read_time 17.519
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 6:48:50 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.046 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 43.019 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 31s
103 actionable tasks: 67 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/vv77cetf3vcbq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #480
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/480/display/redirect?page=changes>
Changes:
[github] [BEAM-3288] Add suggested fix to error message (#11622)
------------------------------------------
[...truncated 292.22 KB...]
May 09, 2020 12:59:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 12:59:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 09, 2020 12:59:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 09, 2020 12:59:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 09, 2020 12:59:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 09, 2020 12:59:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 09, 2020 12:59:35 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2020 12:59:35 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2020 12:59:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2020 12:59:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-fO2L8TP_sI7nDrWUAU8q-A.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-iykDQQF4WQ4pq05BJx6IIA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-NTiRPHXn2dcRcWzchYrVNw.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-diG4mMkaK6KiT8qmCSizZA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-VITeXEfYuTBJLOrvnZ1ETQ.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-w38LaOrOhjizqbWdzsmqDA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-rDjMnLd-Qe63GhAQrq17eQ.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-kf9j5etlUWzZsAxWh0ZajQ.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-evSOyi3ZK6D3UeFoTmBrwA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-fy0DBzhAFAHgaFL20xllVA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-TwsF_xTa9fJ1lAtFudSgcg.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-w3hqHzRBmFiigRuA2cT0ew.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-2tNTNUxeTZ0acyVO33EwiA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-CYTyPdFsTt7qdtiesWC3cg.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-kp04iAfJvBTcmaZKDT4Z9g.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-YGB_w5HXIit7CITcjnfwNA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-A9ViIs9Sw40xWK1L-JtDNA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-D236gpi-6LzfeUxJh3lZBA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-WRF2x1Fdo123iXqiQP7G5g.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-dBSKWNe9PPilqGW-LOquHA.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-SLI13Fg2CpPj2Ps9OZYJbw.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ik3Gd0DDcZSv6zivzqaoHQ.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-AjsD4jhuckItHFCgM_fugg.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-O40LtCzuaHXrhWX6RW5V3A.jar
May 09, 2020 12:59:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-AeodohJPdVd6IcwzbRtvgg.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-DHPCM_Y1xN3ms3xmLgEAHw.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-4Yp1IfYO5A73cp1VMY9xDw.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9ayIDlig2nivBAtcrY3OsQ.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-5MRXFTXJuosNNdy0LHuYmQ.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9ayIDlig2nivBAtcrY3OsQ.jar
May 09, 2020 12:59:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-qMY_4e2eYH9WYObJ78Qdfw.jar
May 09, 2020 12:59:38 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9ayIDlig2nivBAtcrY3OsQ.jar&uploadType=resumable&upload_id=AAANsUnd3dgz4h9PKDXLjgOjFhd7XQX8olD61cfZkqDW8RUnr20Z6Xf9KBuD3605HiGpOODGIR70T0vrqfM3l_wN0HzEnZbbWw.
May 09, 2020 12:59:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 09, 2020 12:59:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9ayIDlig2nivBAtcrY3OsQ.jar
May 09, 2020 12:59:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 7 seconds
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2020 12:59:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash 7EhWEU1Kv6ik3-rcFTuCTQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7EhWEU1Kv6ik3-rcFTuCTQ.pb
May 09, 2020 12:59:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 09, 2020 12:59:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-08_17_59_44-9461068024627000779?project=apache-beam-testing
May 09, 2020 12:59:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-08_17_59_44-9461068024627000779
May 09, 2020 12:59:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-08_17_59_44-9461068024627000779
May 09, 2020 12:59:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T00:59:44.236Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 09, 2020 12:59:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:47.643Z: Checking permissions granted to controller Service Account.
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:52.301Z: Worker configuration: n1-standard-1 in us-central1-a.
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:52.952Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:52.992Z: Expanding GroupByKey operations into optimizable parts.
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.029Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.118Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2020 12:59:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.156Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 09, 2020 12:59:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.191Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 09, 2020 12:59:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.233Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 09, 2020 12:59:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.614Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 12:59:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T00:59:53.685Z: Starting 5 workers in us-central1-a...
May 09, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-09T01:00:19.690Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:00:19.939Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 09, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:00:19.984Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 09, 2020 1:00:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:00:25.573Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 09, 2020 1:00:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:00:38.351Z: Workers have started successfully.
May 09, 2020 1:00:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:00:38.390Z: Workers have started successfully.
May 09, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:01:14.615Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 09, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:01:15.053Z: Cleaning up.
May 09, 2020 1:01:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:01:15.179Z: Stopping worker pool...
May 09, 2020 1:03:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:03:30.025Z: Autoscaling: Resized worker pool from 5 to 0.
May 09, 2020 1:03:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-09T01:03:30.099Z: Worker pool stopped.
May 09, 2020 1:03:38 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-08_17_59_44-9461068024627000779 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0da71e1e-37e7-40f9-93b8-fb8d9774512e and timestamp: 2020-05-09T01:03:38.637000000Z:
Metric: Value:
read_time 18.304
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 09, 2020 1:03:39 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 12.927 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 18s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/knp7pbq33rcv4
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #479
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/479/display/redirect?page=changes>
Changes:
[nielm] Remove spurious error message.
[alxavier] [BEAM-9911]Replace SpannerIO.write latency counter to distribution
[kcweaver] [BEAM-4782] Remove workaround in Python multimap tests.
[kamil.wasilewski] [BEAM-8132, BEAM-8133] Apply InfluxDB pipeline options in Load Tests and
[kamil.wasilewski] [BEAM-8132, BEAM-8133] Assume no trailing slash after a hostname
[iemejia] [BEAM-2530] Compile and run tests on java 11 for Precommit portability
[github] [BEAM-9883] Refactor SDF test restrictions. (#11605)
------------------------------------------
[...truncated 293.79 KB...]
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 08, 2020 6:45:32 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 08, 2020 6:45:33 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 08, 2020 6:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 08, 2020 6:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 08, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-xEJLecW_ERWfqEkYbxMPLA.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-D-r7Mg4bfO30eCfsQx41Rw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-bM1fb5ITYKEK4WWa2KfOvw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-_G4z3-96D_0tBX9ncgmAxQ.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-1E86_C0tuXa_xnoXF2nzzg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-CafohyUUxTm_9zD9b09q8A.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-veIbo6doz7alLrTpbkrW9Q.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ANArtb5cfOqxIuUzgpLD9g.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-si58MHRl1jxSNXeAxvqN2w.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-0YrhMsA1hSQ4jpjn48Ekgw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-_AEIGnuSTxegyf1QtGTR8w.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-KXtsvONRR6zvLrIiFAy4Ng.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-9tCWFSlXjY-Rj-MwHV3FOg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Y3khMtM5lbvRtO6hgZ4VBg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-acbRvX55v4IfV851YSQIQw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-XxLl-wo-XSaI8uVXPcemSw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-fni7WJLu-vl49U3qugNVBw.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-EEsNCCzsTBtGBxogAL5_wA.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-o2z3gl0VXulZLQ-K_RAtAQ.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-FeQCWqPQF6zcC8POdqetFg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tW69PdPeAE5R41X1ccULmg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-OYIHIA-i3fxeh59NxSNIWg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kg_tdLrEypnd9jnH_cybIg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-7JU5Xze3cOufeIxH0QXnGg.jar
May 08, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT--xmkn6wiucbxJxuvNkjLfA.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-YSTGDYMdh-awhCZ2XcaQ7g.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-KHi7u_ltrtyfJer9AcGwBw.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-fbvccFe_eY_HHbZ5605yjA.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bp8Rt4qGv2ecNspNXd6bEQ.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-nLj7CX-0lE6_IGIfH-2h1A.jar
May 08, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bp8Rt4qGv2ecNspNXd6bEQ.jar
May 08, 2020 6:45:36 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bp8Rt4qGv2ecNspNXd6bEQ.jar&uploadType=resumable&upload_id=AAANsUncKlPY6NPgblvJQyT1MqozIq8KZza4VZdrQizJG3tC_GzIF2yBHj5s-1yNPNNHvtE_FdFVC7ksTKZ4WC5NtEbeEE42JQ.
May 08, 2020 6:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 08, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bp8Rt4qGv2ecNspNXd6bEQ.jar
May 08, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 5 seconds
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2020 6:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12098 bytes, hash 7VLogglhftC2EAcqG1F6Jg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7VLogglhftC2EAcqG1F6Jg.pb
May 08, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 08, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-08_11_45_41-7150532961595016929?project=apache-beam-testing
May 08, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-08_11_45_41-7150532961595016929
May 08, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-08_11_45_41-7150532961595016929
May 08, 2020 6:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T18:45:41.155Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 08, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:44.739Z: Checking permissions granted to controller Service Account.
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:50.262Z: Worker configuration: n1-standard-1 in us-central1-a.
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:50.905Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.013Z: Expanding GroupByKey operations into optimizable parts.
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.047Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.137Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.178Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.207Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.235Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.618Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 6:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:45:51.691Z: Starting 5 workers in us-central1-a...
May 08, 2020 6:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T18:46:00.391Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2020 6:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:46:24.504Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 08, 2020 6:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:46:24.529Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 08, 2020 6:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:46:29.893Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 08, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:46:44.366Z: Workers have started successfully.
May 08, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:46:44.400Z: Workers have started successfully.
May 08, 2020 6:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:47:22.837Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:47:23.121Z: Cleaning up.
May 08, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:47:23.206Z: Stopping worker pool...
May 08, 2020 6:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:49:20.127Z: Autoscaling: Resized worker pool from 5 to 0.
May 08, 2020 6:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T18:49:20.181Z: Worker pool stopped.
May 08, 2020 6:49:27 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-08_11_45_41-7150532961595016929 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 95f4153a-68de-4843-953a-188f0ae990eb and timestamp: 2020-05-08T18:49:27.580000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.541
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 6:49:28 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 4 mins 3.254 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 8s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/qffpyp2uz6xpe
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #478
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/478/display/redirect>
Changes:
------------------------------------------
[...truncated 297.13 KB...]
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-JbDVFKRX3ISpKV29Xxs8xA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kCkJgR6EvP7X0bbiUpvxTw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-qzdrIKiw4tJv-tHFG7gX-w.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MxbJZUiCwXQwmACdGH06hg.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-ppj4BivPKiDQB4xYYVK-Cw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-jHciJ-nRIahPDQsCTuPAiA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-odX9T1waon7jDRCk_aD9Rw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-BsKK2_owwnxXHYACLJr9dA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-1Ej4sepoId5G2Y6oSc2NSw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-z2kz-y5mZhB5jmA-NqSSAg.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-HndzuulUpLvI2uuvblkmxQ.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-9dvIwiUvsPpbtpoZxNdmjQ.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LJv_SyONMwR0M-VSlq0juA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-qOAEy-kjbCkPPfKLGfi94Q.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-MajS49A1pLyUtnrfPo-S1g.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-0hRK2ZdjNFAvJezMZ-5DLQ.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-qEt5W9fVxp6IcSybZhqd_Q.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-SiAxAaiVq77dGikna-O0eg.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-P_QIef-R93nKZCVNhZaX3A.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-2Bfy4xHpt9PDpjaxsgyyyA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-3QHPX98lNsDbtr9om-ybFA.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-wLq-f4fp0hktbw0sJ0eBpw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-compiler-3.0.11-bepx6Dd9e13zaxGldAayYQ.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.process/2.1.2/986b38302fa10018d5baccee7f655c31ee9afe5b/de.flapdoodle.embed.process-2.1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.process-2.1.2-YrD3oe9Ff_itNus4wP3waQ.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-h4XA-MN5NozxaCRwqiCMBw.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-OLa7YKJaeGEaXTBtPbU5Vg.jar
May 08, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.mongo/2.2.0/781d14f4e3d9eeb0b4c3e00a4ec165a04b3dc5c4/de.flapdoodle.embed.mongo-2.2.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.mongo-2.2.0-3bAjs_S7MZ7-neWHp_CgmQ.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Acsc2De3Kyq7JAIPufzvTA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fastjson-1.2.49-z0P0MjduZT1wQpv9Ia-IMA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/javax.json-1.0.4-Vphw-XXe7rZpH8ubwCqVVQ.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GXxo4jKRHy3ZOa1Fxwif3g.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ST4-4.0.8-ozr8Njx2YI3X-Kcj8Tfb8Q.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/janino-3.0.11-pubTQ_GoxcNrGgSD5_X7qw.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-bYdncAvXrdQflREcrb8itA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr-runtime-3.5.2-H7uuLLclMCB8ILeXvavQKQ.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-runtime-4.7-t59VAkIGs5viU54ez94MCg.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-4.7-COwoTCR4e1f3u2wy6oa_VA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/mongo-java-driver-3.9.1-SIrJ2zP3qz8Zl5iCpu_6gg.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GXxo4jKRHy3ZOa1Fxwif3g.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/org.abego.treelayout.core-1.0.3-nIzvq2NgpnJWU3DVMR8PPA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-4.0.0-oeIOSKNnBjAj25E3zrfGPA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-K1StyvppMqsAKGHiANtoJQ.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-GCZe6M_ObXgtrK8FkBZFsg.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/icu4j-58.2-YF2KAnaigP9jMsO9JgcRgA.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-platform-4.0.0-6WVyLGquTreCjQNLObQw0g.jar
May 08, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-calcite-1_20_0-0.1-bAqPbFlwv7RdbvHyrAw_yg.jar
May 08, 2020 12:50:45 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GXxo4jKRHy3ZOa1Fxwif3g.jar&uploadType=resumable&upload_id=AAANsUkBUbb3JrJ4pt-VbzrcjgsEEr_4R7JbLpjd9eF7T_XIWdYCrjokjoApxx5MiNHHgQt-oZT6-Qp1euSj9H1J4sI.
May 08, 2020 12:50:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 08, 2020 12:50:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GXxo4jKRHy3ZOa1Fxwif3g.jar
May 08, 2020 12:50:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 163 files cached, 47 files newly uploaded in 9 seconds
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash smY27vwb5NN5L9zkEhK3FA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-smY27vwb5NN5L9zkEhK3FA.pb
May 08, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 08, 2020 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-08_05_50_53-10936299260255719293?project=apache-beam-testing
May 08, 2020 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-08_05_50_53-10936299260255719293
May 08, 2020 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-08_05_50_53-10936299260255719293
May 08, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T12:50:53.945Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 08, 2020 12:51:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:50:57.882Z: Checking permissions granted to controller Service Account.
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:01.979Z: Worker configuration: n1-standard-1 in us-central1-a.
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.722Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.754Z: Expanding GroupByKey operations into optimizable parts.
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.794Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.867Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.899Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.942Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:02.975Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:03.334Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:03.413Z: Starting 5 workers in us-central1-a...
May 08, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T12:51:26.593Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2020 12:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:31.064Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 08, 2020 12:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:31.100Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 08, 2020 12:51:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:36.446Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 08, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:51.568Z: Workers have started successfully.
May 08, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:51:51.602Z: Workers have started successfully.
May 08, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:52:28.969Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:52:29.195Z: Cleaning up.
May 08, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:52:29.280Z: Stopping worker pool...
May 08, 2020 12:54:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:54:15.976Z: Autoscaling: Resized worker pool from 5 to 0.
May 08, 2020 12:54:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T12:54:16.030Z: Worker pool stopped.
May 08, 2020 12:54:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-08_05_50_53-10936299260255719293 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a74d7894-5277-4c6b-b698-0d923dd5b6ba and timestamp: 2020-05-08T12:54:22.939000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.315
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 12:54:23 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 49.963 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/kw6epdmkj6vj6
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #477
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/477/display/redirect?page=changes>
Changes:
[robertwb] Move job server to its own module.
[robertwb] [BEAM-9845] Stage artifacts over expansion service.
[robertwb] [BEAM-9577] Resolve dependencies in Java expansion service.
[robertwb] Start artifact service in expansion service driver. 0;256;0c# Please
[amaliujia] [BEAM-9929] Support UNNEST(array_column) in ZetaSQL.
------------------------------------------
[...truncated 292.75 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 08, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 08, 2020 6:45:30 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 08, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 209 files. Enable logging at DEBUG level to see which files will be staged.
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-xPksxXfo7MJdFvs1GdhU7g.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-JZuYnpNXony-3Lycgt7Y1w.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.22.0-SNAPSHOT-iVVaLuVf5X71bySfchMdTQ.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-53gMwgb7xdca2Cm0zeuDcQ.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-IL_VPMSbvDpPO818jdUjXw.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-HGqFWgbQbGYrIYaT5acuFA.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MkEQ7IxJPMKuj8H3FZ0w3A.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-jWRrWpP7tlEIqxHm3a9_jw.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-n-1LUX6bxSkSCCPNiFHTFQ.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-wGHIszCnDVlBccPaHTu9Iw.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-A-MnOuo19wuZ3qVA3f_eNA.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wbDHi-PuUsrxfhYh6_YAkA.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-aUN79-rwCtpzAy3nMuol3g.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-u3R2BLTEEZ5CdWR_VaSUIg.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT--HQf1a0iW_15-2kTBiX7Cg.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-CetF_W0EeN8a7cogN8iRgw.jar
May 08, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-aatl2GvS_jIBXRkaXJBSOg.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-4NkhJ3CnWlgflqI0EACiVA.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-ZfsSp69Lq_9P0z3_p3IMww.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.22.0-SNAPSHOT-QNV-TU1wXSakNDs2kRzkwA.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-xh462v9BZTrHQGnAE6YVww.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-K1VrIATPCIMm-jrTMpFvGA.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-0COd0iEQQeS7Mpgxxs45NA.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-0EilWCLk-xm98Q9dguxjrw.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-QSNjffC038CdaqhXXb5QCg.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-P1PbsKpymlcKil6GLXeCMg.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-Bpzg28QsgfF5WhUSoRMT-A.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LryAWkkQtFu2rwUhafw6bQ.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-uBj2F9Hw0bZkuZZTqLvFfA.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-28QAjIaYrmBaMVedsn7bcg.jar
May 08, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LryAWkkQtFu2rwUhafw6bQ.jar
May 08, 2020 6:45:33 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LryAWkkQtFu2rwUhafw6bQ.jar&uploadType=resumable&upload_id=AAANsUmuCDyWloCWP9lC0vWx9W2c0PRU7MEIG3WAx32_lO0mVak3xgWhpm6XOJQxInkW1rHJWZtF8Oep9M2cwS_RPGgqg7ZxqA.
May 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 08, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-LryAWkkQtFu2rwUhafw6bQ.jar
May 08, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 31 files newly uploaded in 8 seconds
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash 70h9-qEBVhzr3DUyi0CH_g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-70h9-qEBVhzr3DUyi0CH_g.pb
May 08, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 08, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-07_23_45_41-14628073354132792973?project=apache-beam-testing
May 08, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-07_23_45_41-14628073354132792973
May 08, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-07_23_45_41-14628073354132792973
May 08, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T06:45:41.095Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 08, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:45.357Z: Checking permissions granted to controller Service Account.
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:49.504Z: Worker configuration: n1-standard-1 in us-central1-a.
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.126Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.156Z: Expanding GroupByKey operations into optimizable parts.
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.179Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.251Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.310Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.337Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.368Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.665Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:45:50.731Z: Starting 5 workers in us-central1-a...
May 08, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T06:46:17.946Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2020 6:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:46:20.834Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 08, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:46:39.488Z: Workers have started successfully.
May 08, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:46:39.508Z: Workers have started successfully.
May 08, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:47:12.264Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:47:12.519Z: Cleaning up.
May 08, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:47:12.598Z: Stopping worker pool...
May 08, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:49:11.642Z: Autoscaling: Resized worker pool from 5 to 0.
May 08, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T06:49:11.679Z: Worker pool stopped.
May 08, 2020 6:49:17 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-07_23_45_41-14628073354132792973 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7ca8fe27-eb70-4278-bfef-ecb4376c562a and timestamp: 2020-05-08T06:49:17.529000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.329
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 6:49:17 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 56.557 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 0s
103 actionable tasks: 69 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/e4fq7oljqyi4o
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #476
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/476/display/redirect?page=changes>
Changes:
[github] [BEAM-9731] passert.Equals: sort output strings for easier reading
[github] Revert "Pass pipeline options from caller to expansion service (#11574)"
[klk] Remove tests of Java on Dataflows Java Runner Harness
[github] [Beam-9679] Add Core Transforms section / Map lesson to the Go SDK katas
------------------------------------------
[...truncated 293.10 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 08, 2020 12:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 08, 2020 12:45:39 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 08, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 08, 2020 12:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 08, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-YEDJDGLRsX5Q7WUnKh7YiQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Lg9s2OeHyi1hVHqwq1aMvA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-LLZbeeVcUqQwvVJ58UEThw.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-8SjdW3aqkpay-20fQIOqMA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tdl_vfUFbJjUsYnb5OANeA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-LN37z9CpHay_ZbWZoUkhOg.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-RD0ZfhCBE2ECJ8-Rcrrw1Q.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-aJakOpZloPP-3u8HWsAQoQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-opunKT1OaIOy_ynUsYOF7w.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-2AEtUxrDQ6-YaI0n367MGQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-rL2biw38K_swZmAy1mkEFw.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-AniQWm7x_hCXYcxc4h3-rQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-gBzVP6jeiCPm8-xZcv0BtA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-2ttEJ1wql780cQ2w_nYfkQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-NykdzjvqfTKJSb06yXT1YQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-txfrKdx22kirwtJRR0_ESg.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-m0UxgGvVGsNw0l929F9jZQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-VCXwdmlBs9VUQ-KZIJRJdQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-YO5ExbZINTXHhEYKGPYABA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-MEpxXr5UOBitGnUTVcCsFQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-9Mj05nmEtLX-sAx1qhybpQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-47Mm6v0Psyxdc0R-lZAJ5A.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-YYl-TTQoxaIZmWOxDD9_OQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-yoeNw0y39aUi_O9HNIOuIQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-VM0WuUf4pq7SBWGDTvchbA.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-dIibmKPxdDME4e18coLxGQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-yoeNw0y39aUi_O9HNIOuIQ.jar
May 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-NjEbzfibIS0Mv43Jcj6YxQ.jar
May 08, 2020 12:45:43 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-yoeNw0y39aUi_O9HNIOuIQ.jar&uploadType=resumable&upload_id=AAANsUmTv_w2kTDqrwwrDJ9N_xufRKeTCBwbPEJdQ3-qNqNYuE7yXk0eWay2M2uf_kqWyER4_8USJ1lDhpuZDmorbB4.
May 08, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 08, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-yoeNw0y39aUi_O9HNIOuIQ.jar
May 08, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2020 12:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash Ch0JtLfIkn4teEKIkd8ovQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Ch0JtLfIkn4teEKIkd8ovQ.pb
May 08, 2020 12:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 08, 2020 12:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-07_17_45_49-10059585340740977791?project=apache-beam-testing
May 08, 2020 12:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-07_17_45_49-10059585340740977791
May 08, 2020 12:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-07_17_45_49-10059585340740977791
May 08, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T00:45:49.364Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 08, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:53.155Z: Checking permissions granted to controller Service Account.
May 08, 2020 12:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:58.559Z: Worker configuration: n1-standard-1 in us-central1-a.
May 08, 2020 12:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.391Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.459Z: Expanding GroupByKey operations into optimizable parts.
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.501Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.600Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.639Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.678Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:45:59.712Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:46:00.213Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:46:00.300Z: Starting 5 workers in us-central1-a...
May 08, 2020 12:46:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-08T00:46:04.219Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:46:38.251Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 08, 2020 12:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:46:57.710Z: Workers have started successfully.
May 08, 2020 12:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:46:57.744Z: Workers have started successfully.
May 08, 2020 12:47:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:47:36.897Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 08, 2020 12:47:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:47:37.092Z: Cleaning up.
May 08, 2020 12:47:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:47:37.186Z: Stopping worker pool...
May 08, 2020 12:49:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:49:22.973Z: Autoscaling: Resized worker pool from 5 to 0.
May 08, 2020 12:49:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-08T00:49:23.068Z: Worker pool stopped.
May 08, 2020 12:49:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-07_17_45_49-10059585340740977791 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d90bc60a-3df8-4af4-80f2-698591ec2373 and timestamp: 2020-05-08T00:49:30.359000000Z:
Metric: Value:
fields_read 4375276.0
read_time 22.551
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 08, 2020 12:49:30 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 4 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 0.603 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/xkozydhajq4bg
Stopped 3 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #475
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/475/display/redirect?page=changes>
Changes:
[mxm] [BEAM-8742] Add stateful and timely processing benchmarks
[mxm] [BEAM-9900] Remove shutdownSourcesOnFinalWatermark flag
[github] [BEAM-9913] remove license dir if exists (#11630)
[github] Merge pull request #11624 from [BEAM-9767] Make streaming_wordcount use
------------------------------------------
[...truncated 294.15 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
May 07, 2020 7:07:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 07, 2020 7:07:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 07, 2020 7:07:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
May 07, 2020 7:07:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 07, 2020 7:07:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 7:07:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 7:07:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 07, 2020 7:07:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 7:07:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 07, 2020 7:07:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 07, 2020 7:07:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 07, 2020 7:07:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 07, 2020 7:07:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 07, 2020 7:07:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 07, 2020 7:07:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-xN1umnhhiPmtb42NsJ0oyA.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-eJnRheLKeHKh1rcCFIYHTg.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-OC1ABXYUfTCObNHFC2KzhQ.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-WEXN5rskbuHgzdlSz1T0qg.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-GtbakyrJtw2QnQ_uAEPFtA.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VbRJELyq5jxdwNqytRCmZg.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tFU3CMw1Z6lihGn7Emliiw.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-VO6yH_QhQNUaGQPYDpzm0w.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-yWourwIvsGPc71fMTjmgVA.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-MbgWKeRsnWFZH4XHwvFu8Q.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-_PwUOODN3BujJjCX0h7pgw.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-y0qci06yQDxApqmj6ePWIw.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-AYZZs7ruXu2gROl018Li7A.jar
May 07, 2020 7:07:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-IiceSu7TtvFrYph1XohmJQ.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-MxeCo7ShpZmrBRZmEAWZPg.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-s9GZQ0UHzD_hC0fSGLkh9w.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lbp0S_jfJfa1aC_02eZ_0A.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-hBhdzsa8kriEXlvbHF1EGA.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-JQo4lkBc4mHCkNI6pFd9BQ.jar
May 07, 2020 7:07:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-6jK29sJFvXOhgK68PI9DzQ.jar
May 07, 2020 7:07:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-iAFcMqr_sIZKcr3hYFHkYQ.jar
May 07, 2020 7:07:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-RhakRp0qbQZF-TV3rD-JlA.jar
May 07, 2020 7:07:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-gEitpVOSDGDHtEDfcYwuBQ.jar
May 07, 2020 7:07:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Mo2lci2NYh-u3ntry14tKQ.jar
May 07, 2020 7:07:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-jOJesuM70a6evGXt3Le-Iw.jar
May 07, 2020 7:07:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-rD-Qm3D2P660DixGcbI7eQ.jar
May 07, 2020 7:07:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-qdbB4HEsHtmnj1i3n7UdRw.jar
May 07, 2020 7:07:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 180 files cached, 27 files newly uploaded in 13 seconds
May 07, 2020 7:07:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 07, 2020 7:07:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 07, 2020 7:07:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 07, 2020 7:07:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 07, 2020 7:07:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 07, 2020 7:07:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12098 bytes, hash ZH33mmg6P0-VcwfydL6Ocw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZH33mmg6P0-VcwfydL6Ocw.pb
May 07, 2020 7:07:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 07, 2020 7:07:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-07_12_07_42-11665782744935937964?project=apache-beam-testing
May 07, 2020 7:07:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-07_12_07_42-11665782744935937964
May 07, 2020 7:07:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-07_12_07_42-11665782744935937964
May 07, 2020 7:07:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T19:07:42.893Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 07, 2020 7:07:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:47.567Z: Checking permissions granted to controller Service Account.
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:51.286Z: Worker configuration: n1-standard-1 in us-central1-a.
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:51.991Z: Expanding CoGroupByKey operations into optimizable parts.
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.032Z: Expanding GroupByKey operations into optimizable parts.
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.150Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.235Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.270Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.301Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.335Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.750Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 7:07:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:07:52.818Z: Starting 5 workers in us-central1-a...
May 07, 2020 7:08:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:08:22.119Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 07, 2020 7:08:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T19:08:22.717Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2020 7:08:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:08:40.115Z: Workers have started successfully.
May 07, 2020 7:08:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:08:40.148Z: Workers have started successfully.
May 07, 2020 7:09:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:09:13.397Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 7:09:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:09:13.637Z: Cleaning up.
May 07, 2020 7:09:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:09:13.751Z: Stopping worker pool...
May 07, 2020 7:11:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:11:28.573Z: Autoscaling: Resized worker pool from 5 to 0.
May 07, 2020 7:11:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T19:11:28.618Z: Worker pool stopped.
May 07, 2020 7:11:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-07_12_07_42-11665782744935937964 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 624509ad-7537-42a7-b58a-50929f12fb5a and timestamp: 2020-05-07T19:11:33.970000000Z:
Metric: Value:
read_time 14.557
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 7:11:35 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.108 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.087 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 4 mins 53.804 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 48s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/72e5tzt2al5hq
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #474
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/474/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-8944] Improve UnboundedThreadPoolExecutor performance
[lcwik] fixup! Fix lint
[github] [BEAM-8134] Grafana dashboards for Load Tests and IO IT Performance
[github] [BEAM-8133] Push ioit tests metrics to influxdb
------------------------------------------
[...truncated 289.30 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 07, 2020 1:37:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 07, 2020 1:37:52 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 07, 2020 1:37:52 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-E3KAhR_BOBB5-dWlXxOjNw.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Z3wVXK1Gflrg7X4H5hCtvQ.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-NhlcuMwNKjFe1G8hee__vA.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-He6HiqEWC1jt3A8vWuAR_g.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-oLY1WYLxhktO0hJTPx1I1A.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-MDzjB86v9E4cN7vFZq3R4w.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-fPFd5bD59WXg4KYYJgSNKg.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tSoIe5ITHRodws8-_OCb2g.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-HGYF1-bNCPJ5UcO4Trld3g.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-rh0eOL0A37jQyQQeMswiFA.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-DwmB1wrxBBTPlcKw7gK0zw.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-vqdnYUG-FKDDeJrFbhVM9A.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-fZ87CfSaRD5iAs2gGZiHwA.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-mcMYGxb2UMNkoJLktkSFvA.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-ztOGJIbz4eS3QkaUpp9JUg.jar
May 07, 2020 1:37:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-OfsxtRormzpKLVHEHUzDmQ.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-kEVd3G1usNSGNukoh3WEwQ.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-XpHUiEeIeaEwpVJodPwoRg.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-KjwMkeGCE1GmQYNf25WEoA.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-qh9k64jptFeg9EmDcHq8sw.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-H7Gnw_SttByIriEGY5k2eA.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-PT_CSb2TFjFoqMcMTgo_eg.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JxS-9plt9WrHG0FxR7vDzQ.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-pZYrnH2T88m1QE288ae8UA.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-w0Vq7UZ8RTS1HiftDFuokQ.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-MG9uk5v8HKYlbGezN8zXfA.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-nYw3P0Pz4hVCYkRcnI20UQ.jar
May 07, 2020 1:37:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-w0Vq7UZ8RTS1HiftDFuokQ.jar
May 07, 2020 1:37:55 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-w0Vq7UZ8RTS1HiftDFuokQ.jar&uploadType=resumable&upload_id=AAANsUkruOxnynkFjSUZ9mhTzFTGoJa_dFKXkIVKTkyJzI8C15CmRiG7HXw_sEHQQwnm0Ct32oyxuEVUISf5HXuIJQ.
May 07, 2020 1:37:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 07, 2020 1:38:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-w0Vq7UZ8RTS1HiftDFuokQ.jar
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 07, 2020 1:38:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 07, 2020 1:38:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash FEcCQ11bpFIeWSoNEbrxsg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-FEcCQ11bpFIeWSoNEbrxsg.pb
May 07, 2020 1:38:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 07, 2020 1:38:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-07_06_38_03-14464948940127922439?project=apache-beam-testing
May 07, 2020 1:38:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-07_06_38_03-14464948940127922439
May 07, 2020 1:38:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-07_06_38_03-14464948940127922439
May 07, 2020 1:38:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T13:38:03.639Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 07, 2020 1:38:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:06.797Z: Checking permissions granted to controller Service Account.
May 07, 2020 1:38:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:10.826Z: Worker configuration: n1-standard-1 in us-central1-a.
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.569Z: Expanding CoGroupByKey operations into optimizable parts.
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.706Z: Expanding GroupByKey operations into optimizable parts.
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.744Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.822Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.858Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.899Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:11.933Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:12.438Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 1:38:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:12.516Z: Starting 5 workers in us-central1-a...
May 07, 2020 1:38:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T13:38:35.012Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2020 1:38:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:39.124Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 07, 2020 1:38:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:58.054Z: Workers have started successfully.
May 07, 2020 1:38:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:38:58.090Z: Workers have started successfully.
May 07, 2020 1:39:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:39:32.253Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 1:39:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:39:32.453Z: Cleaning up.
May 07, 2020 1:39:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:39:32.553Z: Stopping worker pool...
May 07, 2020 1:41:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:41:22.874Z: Autoscaling: Resized worker pool from 5 to 0.
May 07, 2020 1:41:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T13:41:22.916Z: Worker pool stopped.
May 07, 2020 1:41:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-07_06_38_03-14464948940127922439 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 083df2e9-d11c-4b15-af19-8b1b96e639db and timestamp: 2020-05-07T13:41:28.084000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.923
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 1:41:28 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.017 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 44.241 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 53s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/zcgptyo4lhg2w
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #473
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/473/display/redirect?page=changes>
Changes:
[robertwb] Fix thread local to be initialized on every thread.
[github] un-duplicate _BEAM_SERVICES
------------------------------------------
[...truncated 287.16 KB...]
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 07, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 07, 2020 6:45:29 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 07, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 07, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 07, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-dSQD9tdUIIGHne0AFapGFA.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-3DAsqVEF5PDuH05jEOCi5Q.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-qRC0JCqzfdJrmpkYGnwwMQ.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-rGw5Zcuw9YyHKS8tbU3KqA.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-K7qx0fYDkxvXOFcT1lxfpg.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-EzImLAXtdcZoldtK9FPzKQ.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-KxGYMhKv71nzg2oE8CSWAA.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-KQXX9iEw2wqWKkuCm_eU-g.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-5ezmiVmo0uDkHiHijpk_3A.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-a3oEgXowaYU2UI9fxG9iiw.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-z3L5e3sR0fXTIQIxerMnGg.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-GwXC9CWkTDwfADwGgNpKfg.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-v4SeaA52Z8TwPgg3fKY7NQ.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-eygdpCPdzrzF36B9Hm4Ynw.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-17QLlBhadqvDRCSoGaDo8w.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Yv1FiRI1d7GaJfllVukhbg.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-owYfbshQGFaQidW1YeODKg.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-GWIRTgwAF7H1xDfdD55-aA.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Jo2QSAmMqc5_XftgQLWd4g.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-yWiqfSo7xhGyzxwHaCaEqQ.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-PJ6A_bW9JJRWHsCAyEQf8A.jar
May 07, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-C5HCJNbr_OgO4TfriKVZng.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F5ae3ypmLAxN9_M_soztKw.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-3Zj12dYFMVLHSi1gGl_E7A.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F5ae3ypmLAxN9_M_soztKw.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-ObJN8nseRWioUrGPpvIe-Q.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-NVE3oSNatNQsXcTgiPMA9Q.jar
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-HDOCdDGvLFKW0vpnaL8wUw.jar
May 07, 2020 6:45:31 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F5ae3ypmLAxN9_M_soztKw.jar&uploadType=resumable&upload_id=AAANsUmq_U0GszO1cp9zVzvx-eTUU05IcWclEUevtOM-spYf-t0HdC5viHx4EpQMWjCGYR0rF626sbqqnIdIAKw_l4_0SdxlAQ.
May 07, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 07, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-F5ae3ypmLAxN9_M_soztKw.jar
May 07, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 07, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12107 bytes, hash YFeZuQh45fw21SNFFsh2FQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-YFeZuQh45fw21SNFFsh2FQ.pb
May 07, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 07, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-06_23_45_37-8207220628551536555?project=apache-beam-testing
May 07, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-06_23_45_37-8207220628551536555
May 07, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-06_23_45_37-8207220628551536555
May 07, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T06:45:37.827Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:41.274Z: Checking permissions granted to controller Service Account.
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:45.938Z: Worker configuration: n1-standard-1 in us-central1-a.
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.672Z: Expanding CoGroupByKey operations into optimizable parts.
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.702Z: Expanding GroupByKey operations into optimizable parts.
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.734Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.831Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.868Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.895Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:46.932Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:47.374Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:45:47.471Z: Starting 5 workers in us-central1-a...
May 07, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T06:46:11.110Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:46:18.508Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 07, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:46:36.142Z: Workers have started successfully.
May 07, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:46:36.168Z: Workers have started successfully.
May 07, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:47:13.122Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:47:13.334Z: Cleaning up.
May 07, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:47:13.431Z: Stopping worker pool...
May 07, 2020 6:49:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:49:04.665Z: Autoscaling: Resized worker pool from 5 to 0.
May 07, 2020 6:49:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T06:49:04.715Z: Worker pool stopped.
May 07, 2020 6:49:11 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-06_23_45_37-8207220628551536555 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1d0b88a0-d897-42f2-b15c-6a3c7b53ea56 and timestamp: 2020-05-07T06:49:11.308000000Z:
Metric: Value:
read_time 18.107
fields_read 4375276.0
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 50.866 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/6o54jlyhmja4u
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #472
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/472/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-7746] Enable mypy type checking for Beam Python code.
[kcweaver] [BEAM-9908] Fix Python build failures in release script.
[github] [BEAM-9840] Support for Parameterized Types when converting from
[kcweaver] Update instructions in release guide.
[iemejia] [BEAM-9836] Exclude Spark runner from UsesKeyInParDo tests
[github] Update sdks/python/mypy.ini
[github] Remove a bunch of spurious warnings in tests. (#11621)
------------------------------------------
[...truncated 289.65 KB...]
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 07, 2020 12:52:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 07, 2020 12:52:10 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 07, 2020 12:52:10 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-s_eDUaG-k7sKV545v0QovA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-0QQ1kl-yX_G-7XdNDyxWmA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-MkzXhR3Ik15LoTVwSUolFA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-AgJcdW0nUfIQlJFyJjq0uw.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ILT-zUkTiPHWVmIW3PG6Wg.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-XS_4KIL1xLh8pir29WsppA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-nVDPFmLZ_mtoTWlskU83jQ.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-NXuUaUFVC2_j8njM_1DmIw.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Xmn1iFBtPjYuDyHin4LQaw.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-EaKkXztRJawnwa0B9_JSQA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-6zz9EAK98DphMEwCMsCLuA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-cDUsaCaM7YS1Wx13l1hR5A.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-octrgMGcCBHYEU4NmUaA0g.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-g8LsK8j-g9DtDiA_NhIzJw.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-1FABHlFHkPVlZ5aop7K5rA.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-jT41OzKwuWuAw0UmiJLUew.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-OwxTyfwlwghlUMR3iwH3Kw.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-fbWwsTDhmymBuoQqK3RA5Q.jar
May 07, 2020 12:52:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-u_TS5_esKEMSsJMJNiNvjg.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-DxXDWNmHBnaVPLZw7tkhlQ.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-TIhKYG9ntKtHjX-LicVfnQ.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Ib7urriNNBotvPQ0MT7Fkg.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-9mplV6vTVBXal7riMcWFEg.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-GhpAE0GJ5a2xzaJARhAP6w.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_OLMBvHCLzBamFrVB9WoNA.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_OLMBvHCLzBamFrVB9WoNA.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4iFlGyGDyM1yGZRcS3Ru5Q.jar
May 07, 2020 12:52:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-BrSSqZtZHUI-mfr6IjN46w.jar
May 07, 2020 12:52:13 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_OLMBvHCLzBamFrVB9WoNA.jar&uploadType=resumable&upload_id=AAANsUn3hgxtQOWEPx9pQyrLOdWzp70w9Zy3he9k_lyYMGpxnnaakQ2Avl76r8C2N2EVgd-rGolJQBZEoyblwWNcHyMwJsDF6w.
May 07, 2020 12:52:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 07, 2020 12:52:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-_OLMBvHCLzBamFrVB9WoNA.jar
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 07, 2020 12:52:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash _kOEfNI7n634w3MOid4kZw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_kOEfNI7n634w3MOid4kZw.pb
May 07, 2020 12:52:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 07, 2020 12:52:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-06_17_52_18-14791255471195360607?project=apache-beam-testing
May 07, 2020 12:52:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-06_17_52_18-14791255471195360607
May 07, 2020 12:52:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-06_17_52_18-14791255471195360607
May 07, 2020 12:52:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T00:52:18.330Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 07, 2020 12:52:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:21.494Z: Checking permissions granted to controller Service Account.
May 07, 2020 12:52:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:25.884Z: Worker configuration: n1-standard-1 in us-central1-a.
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.555Z: Expanding CoGroupByKey operations into optimizable parts.
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.588Z: Expanding GroupByKey operations into optimizable parts.
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.622Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.690Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.719Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.749Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:26.777Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:27.150Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 12:52:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:27.220Z: Starting 5 workers in us-central1-a...
May 07, 2020 12:52:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-07T00:52:37.282Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2020 12:52:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:52:56.293Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 07, 2020 12:53:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:53:17.957Z: Workers have started successfully.
May 07, 2020 12:53:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:53:17.986Z: Workers have started successfully.
May 07, 2020 12:53:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:53:56.686Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 07, 2020 12:53:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:53:56.884Z: Cleaning up.
May 07, 2020 12:53:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:53:56.969Z: Stopping worker pool...
May 07, 2020 12:56:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:56:02.988Z: Autoscaling: Resized worker pool from 5 to 0.
May 07, 2020 12:56:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-07T00:56:03.028Z: Worker pool stopped.
May 07, 2020 12:56:09 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-06_17_52_18-14791255471195360607 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8db1c123-ce5a-482a-a654-9684d45ae078 and timestamp: 2020-05-07T00:56:09.720000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.861
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 4 mins 7.933 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 20s
101 actionable tasks: 75 executed, 26 from cache
Publishing build scan...
https://gradle.com/s/bvz3vq7ztc6pw
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #471
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/471/display/redirect>
Changes:
------------------------------------------
[...truncated 287.85 KB...]
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 06, 2020 6:51:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 06, 2020 6:51:05 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 06, 2020 6:51:05 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-BEjKE92ZJqVo6xtZHm50yw.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-MUagCBY-Z45aFHopZjhidg.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-bcYkjwJ3NlJ74A5Ryb1avw.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-qAJjos9Cj7GsFb9Xz5g_8g.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Cccm1L7JXMIiML1fW1kcfg.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-l6MaMmTIxJbeCpFMflrKpQ.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-fABwiwBIwT7MxCuWY8UhEA.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-UTA0Ng4QV8CnQtBrF9oqKg.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-t8mAP8h8I-FP0H3bb4qjwg.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-HDCfDOPsANXk3siB3CdN5A.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-IUTq95w9O8kKQcmVJ-TTQQ.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-7c_QoGUKi0-azDc1YXHgJA.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-yYjELy7iZHh-ij6s_kQTFA.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-V8XcEVtFSAAxuUJmXaEqXQ.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-lp2MB9F1T98_paIJ_6xRLg.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Scff-T-xxxadUlxoZp62CQ.jar
May 06, 2020 6:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-qreZHiwfHRHNndfzHzEkfw.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-GexTpb9W8O6UFNDLoE5fGw.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-yg8UQl-f4JTbPZo3aVXZOg.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-986Fi76tYwYpY-xsrIcV9w.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-6LClIK2TcBllk0hBlGOTpA.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-XtWe8HfyKSPdX-Z5w-GqZw.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-bElZ4R4J5aCiAtZdXvn9ig.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rfDpUcXLYW-3LDD8dUmvHQ.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rfDpUcXLYW-3LDD8dUmvHQ.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-uJ5ub35riOFJyge19e4gaA.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-8oFb33Re1p406129poL1cw.jar
May 06, 2020 6:51:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-uGB5GqH6cPzwFif32YBq8Q.jar
May 06, 2020 6:51:08 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rfDpUcXLYW-3LDD8dUmvHQ.jar&uploadType=resumable&upload_id=AAANsUkOx9WtKR1uE39d2vOiA1sXE7tAvNajl8k-lOtjJXUva8q9oQ29pwlyqTXOK_SXuX3lxJrnIseyWHc_-dZk9zM.
May 06, 2020 6:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 06, 2020 6:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rfDpUcXLYW-3LDD8dUmvHQ.jar
May 06, 2020 6:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash -2MDOUv368GEBSv14G0s8g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--2MDOUv368GEBSv14G0s8g.pb
May 06, 2020 6:51:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 06, 2020 6:51:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-06_11_51_13-18334159344563735275?project=apache-beam-testing
May 06, 2020 6:51:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-06_11_51_13-18334159344563735275
May 06, 2020 6:51:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-06_11_51_13-18334159344563735275
May 06, 2020 6:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T18:51:13.883Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 06, 2020 6:51:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:18.202Z: Checking permissions granted to controller Service Account.
May 06, 2020 6:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:23.289Z: Worker configuration: n1-standard-1 in us-central1-a.
May 06, 2020 6:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.060Z: Expanding CoGroupByKey operations into optimizable parts.
May 06, 2020 6:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.202Z: Expanding GroupByKey operations into optimizable parts.
May 06, 2020 6:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.254Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 06, 2020 6:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.381Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 06, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.425Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 06, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.460Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 06, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:24.500Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 06, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:25.003Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:51:25.100Z: Starting 5 workers in us-central1-a...
May 06, 2020 6:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T18:51:40.871Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 06, 2020 6:52:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:01.115Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 06, 2020 6:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:17.899Z: Workers have started successfully.
May 06, 2020 6:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:17.939Z: Workers have started successfully.
May 06, 2020 6:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:51.628Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 6:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:52.043Z: Cleaning up.
May 06, 2020 6:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:52:52.184Z: Stopping worker pool...
May 06, 2020 6:54:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:54:47.077Z: Autoscaling: Resized worker pool from 5 to 0.
May 06, 2020 6:54:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T18:54:47.133Z: Worker pool stopped.
May 06, 2020 6:54:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-06_11_51_13-18334159344563735275 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): acbe95eb-6fda-4d15-a5b9-e2ca78d8a92a and timestamp: 2020-05-06T18:54:52.960000000Z:
Metric: Value:
read_time 14.429
fields_read 4375276.0
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 56.142 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 0s
101 actionable tasks: 70 executed, 31 from cache
Publishing build scan...
https://gradle.com/s/hvzkl5tk4zatu
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #470
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/470/display/redirect?page=changes>
Changes:
[je.ik] Disable two flaky tests (BEAM-8035, BEAM-9164)
[je.ik] [BEAM-9888] Drop data based on input watermark in
------------------------------------------
[...truncated 288.51 KB...]
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 06, 2020 12:54:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 06, 2020 12:54:11 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 06, 2020 12:54:11 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-yMkteCZhXJ1Lm4VvlsGrWA.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-gFdXs1pUGDg--HKqrM2AVw.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-e7NKU5MhJL0cjC3OHwsJVQ.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-xnLHVKJaFFrBEQZOI-Ggkg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-TVv5rSiQhHUb9c4aHhigeg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-65fTDqi5wi40OtMD_zo4BQ.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-oOsfhvyCSip1KznVQRb2wQ.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-WMXrOYjQCwhGpzq3GcP6lQ.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-so1oc3Z5FW1Wr8cuKgG98Q.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-wHUHaBXCXGEZRW6Nh8SDgg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-hUaeb3Yx2SNGawHBPmnP7g.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-JTZY6xq7458Cwf8Wdgf1vg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-afItDVBrdRmbGTx4s3gZPQ.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-PR1nTDqV3TuWe8ZMiScvQg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-P9FS9zubqO0AZNgpvyT-cg.jar
May 06, 2020 12:54:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-eKBboQx5SBnxe_FKVIG_AA.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tJx5Bec0ll8Taiq_nMrCEQ.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-jd8mGC8eytQj0miasI-kdg.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-X0Wxq3zKxnVB9quhJ03K2Q.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-BwCIMiO-wN38UBeKpF44nA.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT--mQ5utZ3QXFOfWsrvOlXyw.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Ew8cBeS-G_iwYvs-gr-AOg.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-yliD_hGz4PLGYN-YFsEF6Q.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-BsqoIDdCaRbjspu0w4U2rw.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-bTgfpfVkr_AKwtO2vrwkAw.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-n6TciWn2WsvnrC6VAAU0uA.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-UAVMXJiXtyfuFxiaPo862g.jar
May 06, 2020 12:54:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-UAVMXJiXtyfuFxiaPo862g.jar
May 06, 2020 12:54:14 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-UAVMXJiXtyfuFxiaPo862g.jar&uploadType=resumable&upload_id=AAANsUk8HMnZZo2hcab5TfoqkFfMolI5X6u1Lu4Omz9z0R4Z4e9aUUFFYG-1N13TFiJTqiq1XDKYykgVPEnIrsI6z2k.
May 06, 2020 12:54:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 06, 2020 12:54:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-UAVMXJiXtyfuFxiaPo862g.jar
May 06, 2020 12:54:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 06, 2020 12:54:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash mZpKSZlfkIDuXr5Et0E7Vw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mZpKSZlfkIDuXr5Et0E7Vw.pb
May 06, 2020 12:54:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 06, 2020 12:54:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-06_05_54_20-18253964170398621167?project=apache-beam-testing
May 06, 2020 12:54:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-06_05_54_20-18253964170398621167
May 06, 2020 12:54:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-06_05_54_20-18253964170398621167
May 06, 2020 12:54:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T12:54:20.286Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 06, 2020 12:54:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:23.086Z: Checking permissions granted to controller Service Account.
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:27.034Z: Worker configuration: n1-standard-1 in us-central1-a.
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:27.988Z: Expanding CoGroupByKey operations into optimizable parts.
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.032Z: Expanding GroupByKey operations into optimizable parts.
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.074Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.162Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.198Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.233Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.271Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 06, 2020 12:54:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.887Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 12:54:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:28.973Z: Starting 5 workers in us-central1-a...
May 06, 2020 12:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T12:54:45.593Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 06, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:54:58.479Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 06, 2020 12:55:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:55:17.863Z: Workers have started successfully.
May 06, 2020 12:55:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:55:17.899Z: Workers have started successfully.
May 06, 2020 12:55:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:55:50.812Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 12:55:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:55:51.014Z: Cleaning up.
May 06, 2020 12:55:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:55:51.095Z: Stopping worker pool...
May 06, 2020 12:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:57:53.286Z: Autoscaling: Resized worker pool from 5 to 0.
May 06, 2020 12:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T12:57:53.330Z: Worker pool stopped.
May 06, 2020 12:58:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-06_05_54_20-18253964170398621167 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2818c587-8e0c-4c07-9198-7792a7554bc4 and timestamp: 2020-05-06T12:58:01.049000000Z:
Metric: Value:
read_time 15.015
fields_read 4375276.0
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 58.738 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 7s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/6yyfxqz3ltbu4
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #469
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/469/display/redirect>
Changes:
------------------------------------------
[...truncated 296.08 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 06, 2020 6:50:12 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 6:50:12 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 06, 2020 6:50:12 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 6:50:12 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 06, 2020 6:50:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 06, 2020 6:50:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 06, 2020 6:50:15 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 06, 2020 6:50:17 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 06, 2020 6:50:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 06, 2020 6:50:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-8e-fJ69N7gmoPzannAfQQA.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-C0UMRVIyAculLUVqdngG2g.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-gxFf1H9WVsLOM6eJK5IuOA.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-1mRpVgkPlgJX4mdXgYR45A.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-90cYe5eb1t4VUSoOyIOQIQ.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT--M5O8MihQrvpXs9CLnKSJw.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-MQS6zVHpmuHNNMqA-J1zAg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-hn3dfrNWjl6VR1WbdAzvZg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rCuBh0OvUUcHA2QTt88LSg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-7oWWzt5iqmjY9BFo9X3h4Q.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rCuBh0OvUUcHA2QTt88LSg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-HFSYhaabtwPeuu0trmtr9w.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-yO2PsEZCexxNOYMRO7QC4w.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-fr3esi419wkedi-wfm9EUg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-_HcEUHvIH-sYL6CwTDfp5Q.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-31kAPB7WD1sH2zOmFiS6Mg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-oU-UVBg5VYuTfYtDsNUCPw.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-nXtPi3wogK1DQEGklKZNrw.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Fw3eRwX8Qfp5ipbF2TYu_Q.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-yMV1x-1xch19TgiT9Szgrg.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-tOf5QI96GxBzNmCS7caRKA.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-HIfuf9VNRHfSd0gfW3pd2w.jar
May 06, 2020 6:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-3R-9eUK8WQ41-Q9A5FORIQ.jar
May 06, 2020 6:50:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-WiBEb_Ud80CZ_vxXL4W1uw.jar
May 06, 2020 6:50:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-4DMr8z1s7QAld6YN00bDdw.jar
May 06, 2020 6:50:24 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rCuBh0OvUUcHA2QTt88LSg.jar&uploadType=resumable&upload_id=AAANsUnB8qZtepw912aI8ikKnFlV1USOttDFisycTZZ5UODlUYdY2FkgiEnmX_b06ZiQ-4HY6C-6bLYTTnlmu0BJHTHEPzv2Vw.
May 06, 2020 6:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 06, 2020 6:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-JpAX9GcBd7N__3vHnCTyZw.jar
May 06, 2020 6:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-NDdkLrZYGl6UZW70K8Exog.jar
May 06, 2020 6:50:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-LZEbHnOO_fJOgqFjWim07w.jar
May 06, 2020 6:50:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rCuBh0OvUUcHA2QTt88LSg.jar
May 06, 2020 6:50:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 13 seconds
May 06, 2020 6:50:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 06, 2020 6:50:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 06, 2020 6:50:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 06, 2020 6:50:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 06, 2020 6:50:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 06, 2020 6:50:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash QclA8lNvzTrHBRvUpWtEew> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QclA8lNvzTrHBRvUpWtEew.pb
May 06, 2020 6:50:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 06, 2020 6:50:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-05_23_50_35-16690231102919583289?project=apache-beam-testing
May 06, 2020 6:50:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-05_23_50_35-16690231102919583289
May 06, 2020 6:50:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-05_23_50_35-16690231102919583289
May 06, 2020 6:50:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T06:50:35.123Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 06, 2020 6:50:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:39.753Z: Checking permissions granted to controller Service Account.
May 06, 2020 6:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:43.914Z: Worker configuration: n1-standard-1 in us-central1-a.
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.575Z: Expanding CoGroupByKey operations into optimizable parts.
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.619Z: Expanding GroupByKey operations into optimizable parts.
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.649Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.736Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.764Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.796Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:44.831Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:45.259Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 6:50:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:50:45.352Z: Starting 5 workers in us-central1-a...
May 06, 2020 6:51:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T06:51:11.331Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 06, 2020 6:51:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:51:11.721Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 06, 2020 6:51:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:51:11.756Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 06, 2020 6:51:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:51:17.164Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 06, 2020 6:51:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:51:30.514Z: Workers have started successfully.
May 06, 2020 6:51:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:51:30.546Z: Workers have started successfully.
May 06, 2020 6:52:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:52:09.339Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 6:52:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:52:09.561Z: Cleaning up.
May 06, 2020 6:52:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:52:09.655Z: Stopping worker pool...
May 06, 2020 6:54:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:54:15.114Z: Autoscaling: Resized worker pool from 5 to 0.
May 06, 2020 6:54:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T06:54:15.150Z: Worker pool stopped.
May 06, 2020 6:54:20 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-05_23_50_35-16690231102919583289 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b885e771-9f4b-48cc-8a23-2f83ae3fa36a and timestamp: 2020-05-06T06:54:20.796000000Z:
Metric: Value:
read_time 19.426
fields_read 4375276.0
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.056 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.063 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 4 mins 37.464 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 25s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/voxz2uxet4rts
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #468
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/468/display/redirect?page=changes>
Changes:
[github] Pass pipeline options from caller to expansion service (#11574)
------------------------------------------
[...truncated 286.72 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 06, 2020 12:51:46 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 12:51:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 06, 2020 12:51:46 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 06, 2020 12:51:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 06, 2020 12:51:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 06, 2020 12:51:47 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 06, 2020 12:51:47 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 06, 2020 12:51:48 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 06, 2020 12:51:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 06, 2020 12:51:48 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-C2JvVMP5_Q7U5Jn-6jRxlA.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-MdgZln742VjNe_-ysWTsAw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-PLeLlKopAoLfErOUaZnydg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-9Vrgd_8CmA2sYofjcH_JVw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aDt-aVvaz7jAXUS7BgA7RQ.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-XCsQIB42v7lSx3o6vHNqnw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-A5AHMzfBwfWDH-aSlyDBdw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-08fKFM7MHqGEGVQB_OUhpg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-TwJn2CZOlNroOeXnTHoPWg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-yVOzlag26sf_s0d1ojPKOQ.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-XHgFAjoyEu-qTWca-ZKflw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7CKDzNoMJ2QgfXo7O-6jAQ.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-CfYBuMf7tZyv5C05YMZrsw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-d4VLU_F-uPN06qWFNB8-Kg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-gF4t4rBf8-lzxVNnrhrIPQ.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-yIgsj1KmNR_erDVu38envg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-aGKiw5LYNdPeNoQWcBgg2w.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-rYjEbqOUruOQSJEDeUG1xg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-CS3JMzZ2vsHb9FgMqsARwA.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lwrvoWororm-IlQNrqZusw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-PrCc35n4bZu1b1NaURK-tw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-fDd9BFEryvqnyMjb1XpJRw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kjxLZAPfj-zja9xaZ4pRuw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-KU2BZxPvrCHvv6vg-5yChg.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-zqEPN3CQwWkHbVudnLSRqA.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uiT4K9NA_bSShJVuT4lxnw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uiT4K9NA_bSShJVuT4lxnw.jar
May 06, 2020 12:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-2IhI3i3x88VYrVRZJ-i99g.jar
May 06, 2020 12:51:50 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uiT4K9NA_bSShJVuT4lxnw.jar&uploadType=resumable&upload_id=AAANsUm_NCGa4oQJ_RpplGNb-VO1GCVX8cYFpxlevz7BRl6ntxLGlw2F6bAMMzhH0IbM6-6GYKcGQwk7OTlTgaRq_Q.
May 06, 2020 12:51:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 06, 2020 12:51:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uiT4K9NA_bSShJVuT4lxnw.jar
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 06, 2020 12:51:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash fK-yr9cE9RpYPflXr-bwSg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fK-yr9cE9RpYPflXr-bwSg.pb
May 06, 2020 12:51:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 06, 2020 12:51:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-05_17_51_56-25003644513590973?project=apache-beam-testing
May 06, 2020 12:51:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-05_17_51_56-25003644513590973
May 06, 2020 12:51:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-05_17_51_56-25003644513590973
May 06, 2020 12:51:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T00:51:56.410Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 06, 2020 12:52:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:51:59.770Z: Checking permissions granted to controller Service Account.
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:03.314Z: Worker configuration: n1-standard-1 in us-central1-a.
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.032Z: Expanding CoGroupByKey operations into optimizable parts.
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.075Z: Expanding GroupByKey operations into optimizable parts.
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.112Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.205Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.237Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.277Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.304Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.783Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 12:52:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:04.882Z: Starting 5 workers in us-central1-a...
May 06, 2020 12:52:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-06T00:52:20.600Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 06, 2020 12:52:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:32.977Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 06, 2020 12:52:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:33.013Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 06, 2020 12:52:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:38.421Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 06, 2020 12:52:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:57.140Z: Workers have started successfully.
May 06, 2020 12:52:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:52:57.169Z: Workers have started successfully.
May 06, 2020 12:53:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:53:33.553Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 06, 2020 12:53:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:53:33.779Z: Cleaning up.
May 06, 2020 12:53:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:53:33.874Z: Stopping worker pool...
May 06, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:55:18.559Z: Autoscaling: Resized worker pool from 5 to 0.
May 06, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-06T00:55:18.604Z: Worker pool stopped.
May 06, 2020 12:55:23 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-05_17_51_56-25003644513590973 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a8df7703-7d88-4515-8b47-477f5ac5c08e and timestamp: 2020-05-06T00:55:23.986000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.597
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 44.708 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 48s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/z6zawrqw4nwvg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #467
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/467/display/redirect?page=changes>
Changes:
[apilloud] [BEAM-9661] Fix ORDER BY with LIMIT
[apilloud] [BEAM-9659][BEAM-9657][BEAM-9664] Reject unsupported unnest joins
[github] [BEAM-9650] Add PeriodicSequence generator. (#11477)
------------------------------------------
[...truncated 286.96 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 05, 2020 7:03:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 05, 2020 7:03:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 05, 2020 7:03:24 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 05, 2020 7:03:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 05, 2020 7:03:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-8Eutie8Pqzb-HWig9gOzNw.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-p8DFRh1JimanJjqszpkRWQ.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-zfAaJrzCqzYyD92hP_82Dg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-SwALkuzWz7n1yRPZ0k7BXw.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-65X-GH-KhjA1lu3bjWdMZg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-u8rozmlHAO5Gr728vw3ENg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-HOywV_YRis2sXsRVrdeOPQ.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-IllEcaECmpCojvqlCrAzxg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Ldbz-SjHgFnbHMd0pGLjcg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-HFx2c2KDlg3ZJ2DEQa7afQ.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-eor5YVYcDOWkyTykIGFywA.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-_pbWsDZZRs-eCapVZ8Ta9w.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-LnYRR0TZkuPD1YF838iidw.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-eAkQVpEtoP1Uo8NlsyqMDQ.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-u7wBvdqe0_nVdDAddYxl8w.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-2_P5ICHvc3vz8ZDPDY8O7g.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-qrOZr8oFhv_lpATgYXwUsA.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-l49dc_iJMyrCgWFHGOU9AA.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-jVXy_piqAbyj5OlnP3gG4Q.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-zDFnothm8z_7Tir9qSRqdw.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-pdd8LsbFt78YLg5d2IKLmA.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-jCwMxIqzJWm2_JSdFCUg3w.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-UJz2zJ66GHlMRTrTE9eAbg.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-1c9YJ4ypCfqM3Vgt__Zy4Q.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-oZ-bFTpLzPTgtsP0fNlAYA.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ofzapb6arg12nhiU_U1a9g.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-fWiunX0GT0k7LtX917qzXw.jar
May 05, 2020 7:03:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ofzapb6arg12nhiU_U1a9g.jar
May 05, 2020 7:03:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ofzapb6arg12nhiU_U1a9g.jar&uploadType=resumable&upload_id=AAANsUl04bL3QE_uc7m0SQcdMzoe9CB41rOUpzw9wZu0dlUNfGE6Amz3QN7yfmpsI_22Bdoi1DHqniqUOwe-eRo0oA.
May 05, 2020 7:03:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 05, 2020 7:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ofzapb6arg12nhiU_U1a9g.jar
May 05, 2020 7:03:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 05, 2020 7:03:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash BrJmn1JwXSa60xvIrHzLtw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BrJmn1JwXSa60xvIrHzLtw.pb
May 05, 2020 7:03:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 05, 2020 7:03:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-05_12_03_33-15706477871264664427?project=apache-beam-testing
May 05, 2020 7:03:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-05_12_03_33-15706477871264664427
May 05, 2020 7:03:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-05_12_03_33-15706477871264664427
May 05, 2020 7:03:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T19:03:33.188Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 05, 2020 7:03:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:36.043Z: Checking permissions granted to controller Service Account.
May 05, 2020 7:03:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:40.350Z: Worker configuration: n1-standard-1 in us-central1-a.
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:40.971Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.015Z: Expanding GroupByKey operations into optimizable parts.
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.044Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.135Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.169Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.215Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.246Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.843Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 7:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:03:41.931Z: Starting 5 workers in us-central1-a...
May 05, 2020 7:04:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T19:03:59.932Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2020 7:04:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:04:05.920Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 05, 2020 7:04:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:04:05.974Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 05, 2020 7:04:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:04:11.443Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 05, 2020 7:04:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:04:24.940Z: Workers have started successfully.
May 05, 2020 7:04:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:04:24.978Z: Workers have started successfully.
May 05, 2020 7:05:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:05:00.862Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 7:05:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:05:01.080Z: Cleaning up.
May 05, 2020 7:05:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:05:01.159Z: Stopping worker pool...
May 05, 2020 7:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:07:01.657Z: Autoscaling: Resized worker pool from 5 to 0.
May 05, 2020 7:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T19:07:01.708Z: Worker pool stopped.
May 05, 2020 7:07:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-05_12_03_33-15706477871264664427 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f454fd91-ea64-434f-927a-243d47589253 and timestamp: 2020-05-05T19:07:06.997000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.685
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 51.1 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/suib3cy5akroi
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #466
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/466/display/redirect?page=changes>
Changes:
[aiyangar] Changed sns writes to async
[pawel.pasterz] Update error prone
[pawel.pasterz] Aling EP core and annotations versions
[aiyangar] [BEAM-8542] V2 async changes for SNSIO
------------------------------------------
[...truncated 286.83 KB...]
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 05, 2020 1:09:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 05, 2020 1:09:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 05, 2020 1:09:19 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-okZ4LBDDEU7s59iB3gCu3A.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-ItoCtEoSq15pSDIW0NJakw.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-_HbeZZkXBcN_ER_Tj8FiYQ.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tFWcJct7ufsjbpzMLfCa3Q.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-TW7mDOHhcDRcFvJGuMLDjA.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-EPWzxNZ0auEWs2bBajK9UA.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-2G4i0leL2ebGPplosDqSYg.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-K1_h_DToqf7nVb_zxz4KdA.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-LrQut5m8RS0i87O7I5WQtA.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-OdmhBo5Cp3KYDmAkJRDivw.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-VQ48XP3J-Zsn-r6HsLhHjw.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-QsGYnbE35D9LCXfAXAh7Rg.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-NqsnCryy3xMZ3EE7QTm77g.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-5hfwVCm0sUkMUeqF8IJh9g.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-pgUVubTGLfgmDA9oggap8Q.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests--moJB4-FqzrY9zTTNL92kQ.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-GjWSTJAoxRdZ6d9Tb4EP2A.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-PQnOO-17m14ro3Yfd5vciA.jar
May 05, 2020 1:09:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-CfTOYtoQWhmJ79L8UaiXhg.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-I32pOzk2IyKN1QjJnos5LA.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Sw7a5ei8pD_ANn9xZsDGvQ.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-sjJWzOxjElxPra3itoZX9g.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-U7xBaFNcZq_ExUzYEcuJAg.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-3F8sYoW2ErSjaBsXjYxMdw.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-x3ekPIBW8jsEud7KKsD6bA.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-MTn6mWoiO7StQvGzYfeA2A.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-oqXn3y60lSt5CYCVKV4Bsg.jar
May 05, 2020 1:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-x3ekPIBW8jsEud7KKsD6bA.jar
May 05, 2020 1:09:22 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-x3ekPIBW8jsEud7KKsD6bA.jar&uploadType=resumable&upload_id=AAANsUn26zEc3xiqabZ0txBvZniGBclOCHMvVe7ogWwcdZ5bg3ayMYXhKj1KY5LEzph5HjOPFkuUaz4OdEeRIKfQHrk.
May 05, 2020 1:09:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 05, 2020 1:09:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-x3ekPIBW8jsEud7KKsD6bA.jar
May 05, 2020 1:09:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 05, 2020 1:09:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash -VOigpNOxxtxPGfXdjpI9w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--VOigpNOxxtxPGfXdjpI9w.pb
May 05, 2020 1:09:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 05, 2020 1:09:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-05_06_09_28-2938529398182818484?project=apache-beam-testing
May 05, 2020 1:09:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-05_06_09_28-2938529398182818484
May 05, 2020 1:09:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-05_06_09_28-2938529398182818484
May 05, 2020 1:09:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T13:09:28.899Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 05, 2020 1:09:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:32.347Z: Checking permissions granted to controller Service Account.
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:35.748Z: Worker configuration: n1-standard-1 in us-central1-f.
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.367Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.410Z: Expanding GroupByKey operations into optimizable parts.
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.446Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.530Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.569Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.606Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 05, 2020 1:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:36.639Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 05, 2020 1:09:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:37.074Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 1:09:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:09:37.155Z: Starting 5 workers in us-central1-f...
May 05, 2020 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T13:09:42.922Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2020 1:10:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:04.579Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 05, 2020 1:10:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:28.358Z: Workers have started successfully.
May 05, 2020 1:10:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:28.404Z: Workers have started successfully.
May 05, 2020 1:11:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:58.798Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 1:11:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:59.022Z: Cleaning up.
May 05, 2020 1:11:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:10:59.133Z: Stopping worker pool...
May 05, 2020 1:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:12:46.423Z: Autoscaling: Resized worker pool from 5 to 0.
May 05, 2020 1:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T13:12:46.471Z: Worker pool stopped.
May 05, 2020 1:12:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-05_06_09_28-2938529398182818484 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2eec5244-d638-46f3-ba0f-433437b53e82 and timestamp: 2020-05-05T13:12:54.159000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.802
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 43.506 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/n6glt7wlcfd7w
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #465
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/465/display/redirect?page=changes>
Changes:
[github] Merge pull request #11560 from [BEAM-9886] Auto-inferring project for
------------------------------------------
[...truncated 284.25 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 05, 2020 6:47:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 05, 2020 6:47:26 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 05, 2020 6:47:26 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 05, 2020 6:47:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 05, 2020 6:47:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-TiNkrcq8A3LU9Bny4Yz4Dw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xk6xPf1cBWX-EgHKvukgCQ.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-jCZ2WfM4Ejk8DRo4-mjCBw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-y4gV08xVUMtyTfn7xiZiHQ.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-xNUmI-5METV5uTnbf13o-A.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-lFgVduqPeO7aij1X-xNVDg.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ktMJmfqXW14ptE9Puju2kA.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tACdyd-QKVIBfCkhRagn_g.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-qoj8d8hqUAaS2YVG-7jyog.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-b8NfiwDfnXzz2Ku5YkhrPw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-4tpElNR4B5qYkjfO4Mywnw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aztEALdlSLM-fp51MMSR7A.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-DhH1SvA0RIO19WGkFttfKA.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-cImIbTjkB0kKObLIeJQw_Q.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-zvXO6bydMQ_cf_YF1YILmw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-5e4u421XLUuN1Lzhsri5Xw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-EfsPOhiKSjFvpR9g6cJVqg.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-MbKqmK2o9a3UgVBx8yTGwg.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-7cv-dJiZwm4OY3bfm8b6ng.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-NEdPvj9DJk_ofT6_twxnMw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-h47TXyX52mNkdQqZthn86Q.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-qR0jYae-uaQk0IcYb3MBvA.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-1xz-TyOMdEevqRdgaUUMRw.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-XnFvcahz1KFE_0i-NXCfQA.jar
May 05, 2020 6:47:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-rICPEQn8SmJTwm7ZqcvDlA.jar
May 05, 2020 6:47:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-yLEIvVqM3B3kSV325jIvXg.jar
May 05, 2020 6:47:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--XcpOIk7ZzLTVmt4hEkDDg.jar
May 05, 2020 6:47:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--XcpOIk7ZzLTVmt4hEkDDg.jar
May 05, 2020 6:47:28 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--XcpOIk7ZzLTVmt4hEkDDg.jar&uploadType=resumable&upload_id=AAANsUlufc2v6J1I5SXSbJS0nQTuyvoXJFFFE8Y4gnkp3chYGa9OnsCPpqkVgARrCAcEd0pavkH0WdfxwZCZAguzRfE.
May 05, 2020 6:47:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 05, 2020 6:47:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--XcpOIk7ZzLTVmt4hEkDDg.jar
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 05, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 05, 2020 6:47:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 4-kR0NIsYNrJOrVmGpnEMQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4-kR0NIsYNrJOrVmGpnEMQ.pb
May 05, 2020 6:47:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 05, 2020 6:47:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-04_23_47_34-6543115911758906454?project=apache-beam-testing
May 05, 2020 6:47:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-04_23_47_34-6543115911758906454
May 05, 2020 6:47:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-04_23_47_34-6543115911758906454
May 05, 2020 6:47:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T06:47:34.663Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 05, 2020 6:47:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:37.696Z: Checking permissions granted to controller Service Account.
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:42.194Z: Worker configuration: n1-standard-1 in us-central1-a.
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.182Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.223Z: Expanding GroupByKey operations into optimizable parts.
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.259Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.359Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.390Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.431Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 05, 2020 6:47:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.461Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 05, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.819Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:47:43.887Z: Starting 5 workers in us-central1-a...
May 05, 2020 6:48:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T06:48:01.561Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2020 6:48:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:48:19.833Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 05, 2020 6:48:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:48:19.859Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 05, 2020 6:48:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:48:25.214Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 05, 2020 6:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:48:39.298Z: Workers have started successfully.
May 05, 2020 6:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:48:39.363Z: Workers have started successfully.
May 05, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:49:12.154Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:49:12.375Z: Cleaning up.
May 05, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:49:12.468Z: Stopping worker pool...
May 05, 2020 6:51:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:51:09.999Z: Autoscaling: Resized worker pool from 5 to 0.
May 05, 2020 6:51:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T06:51:10.041Z: Worker pool stopped.
May 05, 2020 6:51:16 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-04_23_47_34-6543115911758906454 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 77425423-8e83-455d-aa47-a36b9e1b8ead and timestamp: 2020-05-05T06:51:16.563000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.083
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 59.11 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/txbpr7nqp7enc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #464
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/464/display/redirect?page=changes>
Changes:
[tudor.marian] Do not attempt to estimate sizes of group-by-key values behind a key;
[tudor.marian] remove comments
[tudor.marian] Make BatchGroupAlsoByWindowViaIteratorsFn extend the
[tudor.marian] revert changes to grouping shuffle reader and iterable like coder
[tudor.marian] fix merge conflicts
[tudor.marian] remove obsolete class
[daniel.o.programmer] [BEAM-9799] Automated validation for RTrackers.
[daniel.o.programmer] [BEAM-9643] Adding Go SDF Documentation.
[github] [BEAM-9880] fix cannot touch issue (#11606)
[daniel.o.programmer] [BEAM-9643] Adjusting signature for TrySplit.
------------------------------------------
[...truncated 284.40 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 05, 2020 12:58:55 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 05, 2020 12:58:56 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 05, 2020 12:58:56 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-hJq2gLi--yqW6OKQ4BD9gg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6vt_U_dKXAH4an5DZEQyIA.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-dKwidOwt2uwduUj9WtYYcg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-ruYVeIeldoFnZ6RP40Jo4A.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-bSDn26r385cAyxLD3fmJqA.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-TeLt1KyYK_m0ESiBWEW6Bw.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-T3E-DM2Og4RGRIEW27Uc0g.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ocanNdID5pGihWFHaqWfNg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-KsLx_DOimghi26tdT75l2g.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-rOuyYpWwBpuEWkas93zshg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-RleWpzoQ92GTQhJHHCjkRg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ILPv0v_Y6-04dwU4Z12KFw.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-fg-P6ouQ_TdLehBIpwdQZA.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tok9RDoGZtWudCD6N9N4Fw.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-lE6U_KDUoFpDdP991i47EQ.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-T2Yij-NYFEqGHjT8Py7LLg.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-gKhvpPwFVLia0TQZ8iVXzA.jar
May 05, 2020 12:58:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-0OcIlpur_WXi97aeYmym0w.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-TSxqQjYv-5Z7isOZg9qDug.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-a_bty76Coci49tpDzHYmyg.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Pbm3nqMWTYoTZAZUiezMOg.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-m2dqdr2Je8hxHpGAjgyleQ.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-AaRu1ChxdK2j_I2XWByOFQ.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-l0cVtaY4oyTuGdDt6Schsw.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-YAgX2B5as8ANKqMxuNiefA.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-bQe9G5RKMUcDdMu1wIm1HA.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fDL5GA3Ekr9GbtTK08cjhA.jar
May 05, 2020 12:58:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fDL5GA3Ekr9GbtTK08cjhA.jar
May 05, 2020 12:58:59 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fDL5GA3Ekr9GbtTK08cjhA.jar&uploadType=resumable&upload_id=AAANsUkjg1SP1nuDy6nKlMWtKx0zUwFfEzcGhXcgPXcahwhI9dab0mhHPtm4s96aS_g2TeZTVMSvAxYYEA07AgHbZ7N1LLCv8A.
May 05, 2020 12:58:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 05, 2020 12:59:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-fDL5GA3Ekr9GbtTK08cjhA.jar
May 05, 2020 12:59:05 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
May 05, 2020 12:59:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash XUTd3EFQx7CMg53Yx1WkiA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XUTd3EFQx7CMg53Yx1WkiA.pb
May 05, 2020 12:59:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 05, 2020 12:59:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-04_17_59_06-6132124471045742545?project=apache-beam-testing
May 05, 2020 12:59:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-04_17_59_06-6132124471045742545
May 05, 2020 12:59:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-04_17_59_06-6132124471045742545
May 05, 2020 12:59:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T00:59:06.930Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 05, 2020 12:59:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:10.086Z: Checking permissions granted to controller Service Account.
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:14.580Z: Worker configuration: n1-standard-1 in us-central1-f.
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.300Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.330Z: Expanding GroupByKey operations into optimizable parts.
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.358Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.415Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.440Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.467Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 05, 2020 12:59:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.491Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 05, 2020 12:59:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.876Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 12:59:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:15.952Z: Starting 5 workers in us-central1-f...
May 05, 2020 12:59:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-05T00:59:27.226Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2020 12:59:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:41.462Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 05, 2020 12:59:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:41.496Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 05, 2020 12:59:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T00:59:46.914Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 05, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:00:01.623Z: Workers have started successfully.
May 05, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:00:01.658Z: Workers have started successfully.
May 05, 2020 1:00:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:00:43.862Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 05, 2020 1:00:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:00:44.078Z: Cleaning up.
May 05, 2020 1:00:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:00:44.163Z: Stopping worker pool...
May 05, 2020 1:02:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:02:40.253Z: Autoscaling: Resized worker pool from 5 to 0.
May 05, 2020 1:02:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-05T01:02:40.297Z: Worker pool stopped.
May 05, 2020 1:02:48 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-04_17_59_06-6132124471045742545 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d32a0111-4e04-49c4-bd78-0f724f482732 and timestamp: 2020-05-05T01:02:48.043000000Z:
Metric: Value:
read_time 21.34
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 59.611 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 3s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/y4u6o7f2w4wua
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #463
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/463/display/redirect?page=changes>
Changes:
[jhnmora000] [BEAM-9418] Support ANY_VALUE aggregation functions
[github] [BEAM-9870] Always generate Dataflow-compatible FnApi protos. (#11593)
[github] Go fmt changes. (#11576)
------------------------------------------
[...truncated 291.34 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 04, 2020 6:47:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 04, 2020 6:47:28 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 04, 2020 6:47:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-gbaNpJp-hEcnT_X0Vyq_pQ.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-JB408_195Z6jPRmfoaFdyg.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-MVSIcu4tslZJmrht8kLzLg.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-WfTDzucbfBdt8F_I2sLeoQ.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-H05yysKfGM06U7I2vm2YHw.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Tuo2_hBVrinJjQwBR3QxGA.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-oJMQDDiaXQNTIcniMRJijQ.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-PgVEjsHqU3uR8bNc9OZs4A.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-8vPwnv5RhCI0uwt__6JkVg.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-dli2HpysGxS0LB2VzOMVpQ.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-mqfjKTes_5MhM-PIVXXzBA.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-O_yNeupW9_w2xXrMNzdLOg.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-11bkoUzjdI7EK2ApYY6zkQ.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-f6ir5or4VDhlQiARWAVmZw.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--_jsCwzFEAjswbkNnNeHTw.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-cHef6uw03htXTvORZ6Lj5w.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-cheQRp675JOdHW49n6Fn1g.jar
May 04, 2020 6:47:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ztaxtV6FHMfIuNYThb2yIA.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-cm27WbeTcZ2BjCVFwBYrvw.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-1cDRzD6yhaWqc5JSmO87_g.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-lvjUemvf-8jiOOn-DZJe4Q.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-xyr_jrTraO9eX1bc5Y96eA.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AX3byIo7Gk63rx0AG7GTuw.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-9BtZZWm6AbyqkKummJIkaA.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-V6Zg0lx5lEIq0VUMaW--kA.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-6Q4Yv_HhAscEX6cI_2lIUw.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ds9GVi005zVksVT0967RfQ.jar
May 04, 2020 6:47:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AX3byIo7Gk63rx0AG7GTuw.jar
May 04, 2020 6:47:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AX3byIo7Gk63rx0AG7GTuw.jar&uploadType=resumable&upload_id=AAANsUk7eyjGn-3wD8yEJQr3OucPMVywlA9yN_7ZjFUQYS5mzAZ_A2VbMvOjTItE7Hyb7KozLfqnPekuBdw4hhUsfJ0.
May 04, 2020 6:47:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 04, 2020 6:47:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AX3byIo7Gk63rx0AG7GTuw.jar
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 04, 2020 6:47:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 04, 2020 6:47:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash IAUqTMp58t16nSYVmKm9GQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IAUqTMp58t16nSYVmKm9GQ.pb
May 04, 2020 6:47:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 04, 2020 6:47:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-04_11_47_37-4658389537550283849?project=apache-beam-testing
May 04, 2020 6:47:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-04_11_47_37-4658389537550283849
May 04, 2020 6:47:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-04_11_47_37-4658389537550283849
May 04, 2020 6:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T18:47:37.571Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 04, 2020 6:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:41.028Z: Checking permissions granted to controller Service Account.
May 04, 2020 6:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:45.652Z: Worker configuration: n1-standard-1 in us-central1-f.
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.385Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.425Z: Expanding GroupByKey operations into optimizable parts.
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.456Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.539Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.572Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.609Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:46.634Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:47.068Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 6:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:47:47.150Z: Starting 5 workers in us-central1-f...
May 04, 2020 6:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T18:48:08.853Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2020 6:48:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:48:16.311Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 04, 2020 6:48:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:48:16.351Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 04, 2020 6:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:48:21.756Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 04, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:48:33.773Z: Workers have started successfully.
May 04, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:48:33.808Z: Workers have started successfully.
May 04, 2020 6:49:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:49:12.116Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:49:12.340Z: Cleaning up.
May 04, 2020 6:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:49:12.410Z: Stopping worker pool...
May 04, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:51:08.845Z: Autoscaling: Resized worker pool from 5 to 0.
May 04, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T18:51:08.883Z: Worker pool stopped.
May 04, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-04_11_47_37-4658389537550283849 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
Load test results for test (ID): 21404155-6c61-4982-9764-5b14f066c76e and timestamp: 2020-05-04T18:51:16.823000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.878
3 tests completed, 2 failed
Finished generating test XML results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.041 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 57.402 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 58s
101 actionable tasks: 96 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/bhpxqfglvvbac
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #462
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/462/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-8025] Increase the number of retrials en retrial delay in case of
[mxm] [BEAM-9841] Support finite wait on PortableRunner's PipelineResult
[mxm] [BEAM-9874] Support clearing timers in portable batch mode (Spark/Flink)
[echauchot] [BEAM-8025] Trace caught NoHostAvailableException and
------------------------------------------
[...truncated 283.37 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 04, 2020 12:50:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 04, 2020 12:50:11 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 04, 2020 12:50:12 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 04, 2020 12:50:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 04, 2020 12:50:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-svcWYdrtOA_m5yLsAjwuRQ.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-eMY7og7ANoLi-ivGcS7AxQ.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-laNKWIRANcc8egIRkBl1Wg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-OvW53FjGkwPBFXJUNqCqag.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-bWtfMfZpbO3up2l0mzZ2Ug.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ThQEDdCCflMvUHGjwCJfwQ.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-QXSAeKeLhiyHrjCxf4gQbA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-_p6s0MKOXgK8Rn6M-loGaw.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-FGmF7dO0KQfTatFcVsgkjw.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-qtirBUjhGDwnN-6KOI2Xtg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LimSJp9OYESZdR41wiiUHA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-qgPn32-DPQ58YRDvdBhlow.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-0O79d5alq_pJGol4gKdlFQ.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Qc9kXwLlrBLrmXVM2cNfDA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-AysjhHFscCx2_UcXny48Bg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-RUROiJIZ59rAQFQBZ2VhIA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-wXYvNLl8chuOdEq4Yh26VA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-5t3c_u4cXyMLVjeKoFPeWA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-hBmNamnRj2p6MaEBNhI9DA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Lb4j9VBruFbXJxyR0dvSNA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-P0MdX-Unj-AVPl3XoeGhnQ.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-zlmuY6WVqFMPVOzBPL1XlA.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-dE_7Wx56V-w969QrEP_6dg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-oSi00vFXZWYcUYTvBrVUuw.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-o8wN4hIRn6oEoXhRg2u3Tg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-hsC2u8IX_J0je-7IV8ARPg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Y_epP3R6WbON3FdoC866Qg.jar
May 04, 2020 12:50:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-o8wN4hIRn6oEoXhRg2u3Tg.jar
May 04, 2020 12:50:14 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-o8wN4hIRn6oEoXhRg2u3Tg.jar&uploadType=resumable&upload_id=AAANsUkulDp91HdA1CcRU04FK1Pc3mdSAQ0CO6XnSC6S4kVFFOUdObZuddK6CJi5tkBdWOwDQfttzLBfEDPL0aseMZA.
May 04, 2020 12:50:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 04, 2020 12:50:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-o8wN4hIRn6oEoXhRg2u3Tg.jar
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 04, 2020 12:50:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash 7yhwdW6_w082tAzQI5x-Bw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7yhwdW6_w082tAzQI5x-Bw.pb
May 04, 2020 12:50:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 04, 2020 12:50:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-04_05_50_23-3152220166929465999?project=apache-beam-testing
May 04, 2020 12:50:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-04_05_50_23-3152220166929465999
May 04, 2020 12:50:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-04_05_50_23-3152220166929465999
May 04, 2020 12:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T12:50:23.232Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 04, 2020 12:50:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:26.590Z: Checking permissions granted to controller Service Account.
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:30.513Z: Worker configuration: n1-standard-1 in us-central1-f.
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.206Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.248Z: Expanding GroupByKey operations into optimizable parts.
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.288Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.378Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.407Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.446Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 04, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.477Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 04, 2020 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:31.940Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:32.025Z: Starting 5 workers in us-central1-f...
May 04, 2020 12:50:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T12:50:45.528Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2020 12:50:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:56.549Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 04, 2020 12:50:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:50:56.578Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 04, 2020 12:51:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:02.004Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 04, 2020 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:15.383Z: Workers have started successfully.
May 04, 2020 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:15.416Z: Workers have started successfully.
May 04, 2020 12:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:49.096Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 12:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:49.269Z: Cleaning up.
May 04, 2020 12:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:51:49.347Z: Stopping worker pool...
May 04, 2020 12:53:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:53:39.643Z: Autoscaling: Resized worker pool from 5 to 0.
May 04, 2020 12:53:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T12:53:39.686Z: Worker pool stopped.
May 04, 2020 12:53:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-04_05_50_23-3152220166929465999 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 98ee034f-4a5d-4ade-b124-ddba16401049 and timestamp: 2020-05-04T12:53:46.146000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.221
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 42.97 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 42s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/y2u7hibuf6zqc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #461
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/461/display/redirect>
Changes:
------------------------------------------
[...truncated 283.70 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 04, 2020 6:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 04, 2020 6:45:14 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 04, 2020 6:45:14 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 04, 2020 6:45:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 04, 2020 6:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-MtewtW7DyCxOz1N9lNL4YQ.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-SqzpIllIOOaK0gB0NP9xsg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-84pbxsQhTD8XDo0eHEcvQw.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-f4a2-ZalUB6RPHkpb0wsPg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-HeSimhZ6hCqdj5OL1PqHpg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-CJ2xJL-rJl78JEIJjG6Xfw.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-p4LvljHG_v8JQHKWalM_FA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ewCnQ4lYycuvKpX70FflxA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-c0lk7YonblWGeR-idpMYiQ.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-GflInqaKbPj9PUVZxyHzEQ.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-rwbULlUJW_-e23AR4rzTfA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-0Hak9tYEjoE-F7y5QfRy_w.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-TtEbtia1cyuSOav7W4y6pQ.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-NW_r2ly1Mp8KkwOI3NJE9Q.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-a7BffJtjR5nERKNRDNUA1w.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-PXMewAELbZq-mMojWhlmKg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-J0GrNc2traYLyHNYVEAEoA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-ycaFk3FqG-C7tp8IE_484A.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-1HDCePvJmNbyFpaeS1M8SA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-B1DC5awSUMlY8SjyvNcU7g.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-IIWcG7yEvdxg5MRO4MkbDw.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-_oWFkgogzL3RCM6xv4LRkA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-w6JHtDhp_RdRcghaCKzjow.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-5qFhD1V1Z0PlWKJ5ppiJTg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-L4mVoH-WoJnG26t8HbPNjA.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-LinwF_deOhayZvD9DPksOg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-apVFP9yCKSsv_J9yW0WVEg.jar
May 04, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-apVFP9yCKSsv_J9yW0WVEg.jar
May 04, 2020 6:45:16 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-apVFP9yCKSsv_J9yW0WVEg.jar&uploadType=resumable&upload_id=AAANsUkZ8VZy3PRW0VHie8OU24pJGbBuZ9bnYViqrWUV8rZ0IW4540V7V3I0SfLbqiXpqoXxG112rYq0QSd4T5dSPReqdVWodQ.
May 04, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 04, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-apVFP9yCKSsv_J9yW0WVEg.jar
May 04, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 04, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash NUuwo66q5o-dDKPJ5sK-_g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-NUuwo66q5o-dDKPJ5sK-_g.pb
May 04, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 04, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-03_23_45_22-5072124688356738667?project=apache-beam-testing
May 04, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-03_23_45_22-5072124688356738667
May 04, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-03_23_45_22-5072124688356738667
May 04, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T06:45:22.728Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 04, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:26.330Z: Checking permissions granted to controller Service Account.
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:31.629Z: Worker configuration: n1-standard-1 in us-central1-f.
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.278Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.317Z: Expanding GroupByKey operations into optimizable parts.
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.359Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.434Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.464Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 04, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.499Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 04, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.537Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 04, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.907Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:45:32.993Z: Starting 5 workers in us-central1-f...
May 04, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T06:45:38.223Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:02.131Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 04, 2020 6:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:17.709Z: Workers have started successfully.
May 04, 2020 6:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:17.748Z: Workers have started successfully.
May 04, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:51.553Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:51.726Z: Cleaning up.
May 04, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:46:51.797Z: Stopping worker pool...
May 04, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:48:17.822Z: Autoscaling: Resized worker pool from 5 to 0.
May 04, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T06:48:17.870Z: Worker pool stopped.
May 04, 2020 6:48:23 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-03_23_45_22-5072124688356738667 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 02df394b-d987-4887-baef-da124640835a and timestamp: 2020-05-04T06:48:23.076000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.464
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 17.756 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 5s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/73l3m67datknw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #460
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/460/display/redirect>
Changes:
------------------------------------------
[...truncated 283.50 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 04, 2020 12:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 04, 2020 12:45:37 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 04, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 04, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 04, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-fXvtYCkTtQmWHxOHSm7LOw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-sezaMkTvuPTGv0w_oMw9Yw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xAlh6jPfPLCKjZbYHUB19Q.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-4n9r_EJjf7ehXGWdekqzGg.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-6K0Gi5M2fD7sMV9uEPthDw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-GvVFZ7i48uS_jT4RoUUFhQ.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-2fQ_2slX142Vc_LQKdmBeg.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Knlt6LPChSZWcURbPanv4Q.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Q5OkAJjs1x5Jk6gkGAug4w.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-id0VYIi6TULtSBwQ5co-3w.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-l5DhxC8v7AMINnPB3_FVIQ.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7UH4HZq2WL0cpkc2cffzig.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-WnxyyPQZPz30XjYeDpFYVg.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-rTP0xCQCeJ94QxGaIpGkOQ.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-oPzTCjw8f9AHg2-Q8GKWTw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-CHuVctixnSeQ5QOBpWLI7A.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-WEQZOQ61T6aQLzRZcO0P5A.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-9lssgwa5U1I3idaom5Z-Sw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-_Tsts_ttJC0IO4TBkVjNoA.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests--g6-ARFeEoKNwUsaW43X5g.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-elEzwNGJgFf4fTd068kblw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-ktTdyeokYfJBNKIx5Q_KFA.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-zVPfQi2gbutKykPjDYiqRw.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-He8Il5EhAhmjtEn8epn-Hg.jar
May 04, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-8S2WRyV8Pkbgk3HUPvctxw.jar
May 04, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-R10HdbxgxyZ8lFZuebQOXA.jar
May 04, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kCbKdAILIC72xpdUK7YP_A.jar
May 04, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kCbKdAILIC72xpdUK7YP_A.jar
May 04, 2020 12:45:40 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kCbKdAILIC72xpdUK7YP_A.jar&uploadType=resumable&upload_id=AAANsUmVjerKbTXwPQAVj7rFlSKO3lvP5QAkHVHV7Ink5ZzYy9xOwxP0wD_Glsd0XOXwcLZ9fkdjfsGvf5cy0kMZWPboX38OSw.
May 04, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 04, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kCbKdAILIC72xpdUK7YP_A.jar
May 04, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 04, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash _yflZG4vv7ZSlIRApVKHTg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_yflZG4vv7ZSlIRApVKHTg.pb
May 04, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 04, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-03_17_45_46-5134874630834513761?project=apache-beam-testing
May 04, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-03_17_45_46-5134874630834513761
May 04, 2020 12:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-03_17_45_46-5134874630834513761
May 04, 2020 12:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T00:45:46.089Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 04, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:49.785Z: Checking permissions granted to controller Service Account.
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:53.941Z: Worker configuration: n1-standard-1 in us-central1-f.
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.676Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.720Z: Expanding GroupByKey operations into optimizable parts.
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.754Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.854Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.882Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.903Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:54.936Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:55.326Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:45:55.399Z: Starting 5 workers in us-central1-f...
May 04, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-04T00:46:14.492Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:46:20.223Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 04, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:46:20.262Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 04, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:46:25.807Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 04, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:46:38.510Z: Workers have started successfully.
May 04, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:46:38.553Z: Workers have started successfully.
May 04, 2020 12:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:47:12.830Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 04, 2020 12:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:47:13.035Z: Cleaning up.
May 04, 2020 12:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:47:13.124Z: Stopping worker pool...
May 04, 2020 12:49:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:49:15.976Z: Autoscaling: Resized worker pool from 5 to 0.
May 04, 2020 12:49:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-04T00:49:16.021Z: Worker pool stopped.
May 04, 2020 12:49:23 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-03_17_45_46-5134874630834513761 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): bdd4150b-e544-499b-ac6f-67f99a7d8895 and timestamp: 2020-05-04T00:49:23.190000000Z:
Metric: Value:
read_time 14.348
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 54.858 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/3rmwdn7emjkto
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #459
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/459/display/redirect>
Changes:
------------------------------------------
[...truncated 284.81 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 03, 2020 6:46:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 03, 2020 6:46:57 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 03, 2020 6:46:57 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 03, 2020 6:46:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 03, 2020 6:46:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-47EAcV5Hv6Nijau3gyYGmw.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-QQy8vE51-63B5knUxGUhSg.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-2QPn880taEzZwXshyUGr0A.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-MqKMRCrk21xz5Z0KqvVoeQ.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-rSwl4sd2k7UnJgF_jLrIYg.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-A-Smz7FWCwDWbjc611SdDA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-F5r-ebvpqpxHiS0cBIwaUQ.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-gvb76P8NA1PX425bkOAZnQ.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Z7yBlv14rD815eQveMVVSw.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests--uVAjVqNz1EmQDYROkVz_g.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-3ZAarMUUJqDLn726KiBUOg.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-wVZbWCw48Zj2-duLyXsrlA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-gJl8wJPtySIEa0Tqu2VS4A.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-TDXHr4wtPZKTDiSpzJE09w.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-_fnDjkXyjjvCDn51tRWjoA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-valtD5jbBsAkpx-2hpFFhA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3l5vrQ0qUltYGjia1m7BaA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-2rrYfocId3-T6Dn2WG71iA.jar
May 03, 2020 6:46:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-VEB7Osl-RtfrXQquW5T6NQ.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-flEY7khYjqbKn1v4lu3grA.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-MzM8pMBw2z61fgrObL5bFA.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-OBhjJz--D_IM8oHt6gviMA.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-jyNtvuIL4uvZobktYMKrwg.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-jyNtvuIL4uvZobktYMKrwg.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-0Xt0A1Yxn3vBdDFbmLQYUw.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-_HorH4GacmE_dqOuImur2Q.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-KZi_QmlK4a07U4p8ds6E_Q.jar
May 03, 2020 6:46:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-SwHdOzqwDz9n5kVuO1Ad1w.jar
May 03, 2020 6:47:00 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-jyNtvuIL4uvZobktYMKrwg.jar&uploadType=resumable&upload_id=AAANsUmIsqq6wWyI22ExgAvVFAyDXARRUziUfHHn0gs4whApWieJMIc86IjSgHQ956sQdZ2fvHAqThOeOjwIiCEGEB7Brdwh1A.
May 03, 2020 6:47:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 03, 2020 6:47:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-jyNtvuIL4uvZobktYMKrwg.jar
May 03, 2020 6:47:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
May 03, 2020 6:47:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash 1vDgKoD8ETiV0vjGgWWlPw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1vDgKoD8ETiV0vjGgWWlPw.pb
May 03, 2020 6:47:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 03, 2020 6:47:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-03_11_47_08-7991212298288359938?project=apache-beam-testing
May 03, 2020 6:47:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-03_11_47_08-7991212298288359938
May 03, 2020 6:47:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-03_11_47_08-7991212298288359938
May 03, 2020 6:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T18:47:08.887Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 03, 2020 6:47:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:12.291Z: Checking permissions granted to controller Service Account.
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:17.394Z: Worker configuration: n1-standard-1 in us-central1-f.
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.051Z: Expanding CoGroupByKey operations into optimizable parts.
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.099Z: Expanding GroupByKey operations into optimizable parts.
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.139Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.221Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.249Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.286Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.321Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:18.944Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 6:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:19.032Z: Starting 5 workers in us-central1-f...
May 03, 2020 6:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T18:47:35.207Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:44.245Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 03, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:44.286Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 03, 2020 6:47:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:47:49.794Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 03, 2020 6:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:48:12.618Z: Workers have started successfully.
May 03, 2020 6:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:48:12.662Z: Workers have started successfully.
May 03, 2020 6:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:48:53.460Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 6:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:48:53.699Z: Cleaning up.
May 03, 2020 6:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:48:53.822Z: Stopping worker pool...
May 03, 2020 6:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:50:59.044Z: Autoscaling: Resized worker pool from 5 to 0.
May 03, 2020 6:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T18:50:59.102Z: Worker pool stopped.
May 03, 2020 6:51:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-03_11_47_08-7991212298288359938 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e60e22eb-6067-4e16-a5b1-effbec5bac54 and timestamp: 2020-05-03T18:51:04.853000000Z:
Metric: Value:
read_time 18.034
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.149 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 16.203 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 17s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/syvssjcxsvsp6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #458
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/458/display/redirect?page=changes>
Changes:
[kcweaver] Refactor timer firing in Flink & Spark portable batch.
[kcweaver] [BEAM-9801] Fire timers set within timers in Spark.
------------------------------------------
[...truncated 284.87 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 03, 2020 12:45:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 03, 2020 12:45:51 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 03, 2020 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 03, 2020 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 03, 2020 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-pXDJdNBhCty_2NOJh01G3Q.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-HOMTwJhWx-e0rIM_7wPI4g.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-d2k-ouaToxJQGDvClLGS3g.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-2jY70n2vMEBvrkSXwwvOaA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-X0f5ZSE--KCaqW9MtYxP8A.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-6NXdwrlAlHCg8uV9GjFAUg.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-cqBofZGQIXYY3rmr2mJEKw.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-7sH_flqQpYPBG2TsXCN51g.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Q1L9jWFVZYYG1IM3fRwHpA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Jp1GlivKuEKaCRYyYrEJsA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-kidPRrRjt_uoa_AUpODnlA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-0z3m0_xrVavybhTAFdqsOw.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-4k9TGN96IyOGwcgWrKgcPA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Rod_KNGkF0ESaL7-hCdVTQ.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-IJmbnNQjISyX7ax8LrNKEg.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-TJf5xFdPNUCXPhRtkad_WQ.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-0dLePdW5HByS2lzEcdMe5w.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-fL504AeTWAfTa92ob2L0Tg.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-StfC3Wl1Z_HPjk_rvJ382A.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-3Q4tX9xLE-X9SeHgduo6Vg.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-6SpFxTZ6QMA05kumatzrcw.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-8zB03oc6St_zxukaFTFSFA.jar
May 03, 2020 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-A2h3fwwOYaOOF5z3Hu7Qzg.jar
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Kf-JBdwuzlG_63BPXG9DJw.jar
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rJzMsv72F49nEsdW8QqeJQ.jar
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-1lKSadqB7WOeSN_k9waW4Q.jar
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rJzMsv72F49nEsdW8QqeJQ.jar
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-EtbOm-YFkxps8r41tzDnfw.jar
May 03, 2020 12:45:53 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rJzMsv72F49nEsdW8QqeJQ.jar&uploadType=resumable&upload_id=AAANsUnpPTl1fasqZFKc8-FbfnLgBlL5P4TRW4LtndlGPGTfte27AFEXc3qnVHYQTn5t2TJHVnlDurqCVrrcagpbzqXB1lPxxQ.
May 03, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 03, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-rJzMsv72F49nEsdW8QqeJQ.jar
May 03, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 03, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash jftCG_4OywXENtBfx9Y4WQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-jftCG_4OywXENtBfx9Y4WQ.pb
May 03, 2020 12:45:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 03, 2020 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-03_05_45_59-12249921515865429916?project=apache-beam-testing
May 03, 2020 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-03_05_45_59-12249921515865429916
May 03, 2020 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-03_05_45_59-12249921515865429916
May 03, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T12:45:59.093Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 03, 2020 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:02.647Z: Checking permissions granted to controller Service Account.
May 03, 2020 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:06.739Z: Worker configuration: n1-standard-1 in us-central1-a.
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.380Z: Expanding CoGroupByKey operations into optimizable parts.
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.424Z: Expanding GroupByKey operations into optimizable parts.
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.458Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.549Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.577Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.617Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:07.662Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:08.050Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:08.133Z: Starting 5 workers in us-central1-a...
May 03, 2020 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T12:46:26.228Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2020 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:31.136Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 03, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:55.674Z: Workers have started successfully.
May 03, 2020 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:46:55.717Z: Workers have started successfully.
May 03, 2020 12:47:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:47:32.628Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 12:47:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:47:32.822Z: Cleaning up.
May 03, 2020 12:47:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:47:32.903Z: Stopping worker pool...
May 03, 2020 12:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:49:24.979Z: Autoscaling: Resized worker pool from 5 to 0.
May 03, 2020 12:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T12:49:25.029Z: Worker pool stopped.
May 03, 2020 12:49:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-03_05_45_59-12249921515865429916 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c6bb2f39-15e7-4bf1-91b7-b11756d8af4b and timestamp: 2020-05-03T12:49:30.704000000Z:
Metric: Value:
read_time 16.019
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 48.158 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
101 actionable tasks: 71 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/4d7lky53oinak
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #457
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/457/display/redirect>
Changes:
------------------------------------------
[...truncated 283.24 KB...]
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 03, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 03, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 03, 2020 6:45:10 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 03, 2020 6:45:10 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 03, 2020 6:45:10 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-KicQVX1HZomZLY2kb-LOUQ.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-ts02WOD1mXmdb033SO9HEw.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-JJDBArApdCPHOSQ1qMZy8A.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-rgTktgmvP3soUicnyKERYw.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-mToF5HLt0KWN1BCoV3zlIQ.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-iMq_ukgtYlXNmyhs8vEK9g.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-zMzNFOpFeqRk-emQe9u0rQ.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-evhzjTSuofFeiht4a4YCow.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-9sg2N22ulWF4vgC5ICulOQ.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-buGcO89k9FB-POvtfPQ8Iw.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-YYE76orckPWe4tyYDGxg2Q.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-i-7m9CtC_-20K1ZgJcNFqw.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ZglDS3JlDp0p03UuFy7-GA.jar
May 03, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT--eHSag_BrU82Pv5RbXuG0w.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-pZM7tYPRkr67r8k1DW5ipA.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-qODsPHH1wq4WXtj3FKJH6Q.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-M_BnsCGQtwmdy6KXxaus4Q.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-qhFzlkph3vqt5aUuL7GYow.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-fe0Mx-Oc5QbTGkem7yjUiA.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-LgoOMWaV6beUrV4Jf12UWw.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-FRpcZUByadE7TmwtcmEQGQ.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-7WzF4Z1ElULyCBacWhsE-w.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-8gTJWU-ikZb7A5Vy3KyFpQ.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-p97yx3e7Y3cLx3mnfAgh9w.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-gx8dz1F_F8Y3EpA8LO1aww.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-gx8dz1F_F8Y3EpA8LO1aww.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-WSwJo3T554OpFpRex3e-BQ.jar
May 03, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ZPw9yvu0D_3CHA7fxOG--w.jar
May 03, 2020 6:45:13 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-gx8dz1F_F8Y3EpA8LO1aww.jar&uploadType=resumable&upload_id=AAANsUlfX4VOh0FVzxxnmfgIIu9V7CPMScNOaBp3fEOd_hwsylkZ8SwlDdM0XjoyuSE61qfEoYXk2CrLFfyyM7coMx0J5KNcYw.
May 03, 2020 6:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 03, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-gx8dz1F_F8Y3EpA8LO1aww.jar
May 03, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 03, 2020 6:45:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash YVqDvsBq8ujx8QQgqaHBTA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-YVqDvsBq8ujx8QQgqaHBTA.pb
May 03, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 03, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-02_23_45_19-12694575653262759842?project=apache-beam-testing
May 03, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-02_23_45_19-12694575653262759842
May 03, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-02_23_45_19-12694575653262759842
May 03, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T06:45:19.758Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 03, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:46.817Z: Checking permissions granted to controller Service Account.
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:50.878Z: Worker configuration: n1-standard-1 in us-central1-f.
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.433Z: Expanding CoGroupByKey operations into optimizable parts.
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.478Z: Expanding GroupByKey operations into optimizable parts.
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.518Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.607Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.682Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.715Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:51.755Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:52.238Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:45:52.316Z: Starting 5 workers in us-central1-f...
May 03, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:46:21.214Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 03, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T06:46:26.862Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:46:38.800Z: Workers have started successfully.
May 03, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:46:38.821Z: Workers have started successfully.
May 03, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:47:15.080Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:47:15.295Z: Cleaning up.
May 03, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:47:15.372Z: Stopping worker pool...
May 03, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:48:50.928Z: Autoscaling: Resized worker pool from 5 to 0.
May 03, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T06:48:50.973Z: Worker pool stopped.
May 03, 2020 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-02_23_45_19-12694575653262759842 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 96b87b78-ebe9-472b-9f37-16f92dde0275 and timestamp: 2020-05-03T06:48:56.440000000Z:
Metric: Value:
read_time 17.818
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 54.845 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 39s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/jahizm3evp626
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not accessed on or after Sun Apr 26 06:44:25 UTC 2020.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.201 secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #456
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/456/display/redirect?page=changes>
Changes:
[github] Merge pull request #11523 from [BEAM-8414] Cleanup Python codebase to
------------------------------------------
[...truncated 283.60 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 03, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 03, 2020 12:45:16 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 03, 2020 12:45:16 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 03, 2020 12:45:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 03, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-NgLGYaBpdrd9GNKQLk-YIQ.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-R8BTVvrO6wIVzeXFbM_anw.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-fMJP7d7i4ooOwAT2LTXB0w.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-rF8pHdEB62I-ibAcxPJgrw.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-A6q0ZYwg6ylzMJ-jKuDqjA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-djw7njMkl2i6kRx9kbqP2A.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-2P00xVywPUn0QUtPEOe-JA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Kd7ngyhuZsLCd3fpF8nWHA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-avbsJbX7CA_akv5djDYUhg.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-tWfKoFCEY7x1YsRqFECeXg.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-onhjtOfr4wRScuf9at1frg.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-gDiG1Zc_pwdERfpLWOm_Vw.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-0ZI9ErRjg9rLA3vcrdqeew.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-NjrpZqWufgiER7WbmzNxNg.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-FgAr7MuY7ewqGMgUajpiQg.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-D1DkemBprHKOfXgb9LRsvQ.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-GAbI6ngLgeg5OEiLGSvbBQ.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-GNr5evLsspLgxq60gRHfBA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-CqAbDZUQDkoCDYv1BRfl5w.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-LMwx0VfwRE24nnVTWPxE_w.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-iNf9IyH3eYV3AjJAI0diYA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-FxsNR0hUh3PqCOpJNn1sVw.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-jpUd85GwyFvXK8dHEEKHGA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-jqDeNR0-tbVkUNvqxAWkNQ.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-fNR3Zf0PV-G4lmY_9ak3-Q.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZB9cQU7s-suDUsqHpmchRA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZB9cQU7s-suDUsqHpmchRA.jar
May 03, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-xzHAAxmZWYwtq_8ded-_Xw.jar
May 03, 2020 12:45:18 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZB9cQU7s-suDUsqHpmchRA.jar&uploadType=resumable&upload_id=AAANsUkGyEHLcVbQfQzZv0JlduvzfyxqFRYEt3JuJltmqxZP-JOCT5nFDzQCTxGgiwuq8HUs01PWJ-LGNSAu0WM2yaY9UJlkRA.
May 03, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 03, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZB9cQU7s-suDUsqHpmchRA.jar
May 03, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 03, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash fhJvCdC0-VfzGVzdi6NgBg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fhJvCdC0-VfzGVzdi6NgBg.pb
May 03, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 03, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-02_17_45_26-1130745998024889789?project=apache-beam-testing
May 03, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-02_17_45_26-1130745998024889789
May 03, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-02_17_45_26-1130745998024889789
May 03, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T00:45:26.318Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 03, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:29.273Z: Checking permissions granted to controller Service Account.
May 03, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.060Z: Worker configuration: n1-standard-1 in us-central1-f.
May 03, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.728Z: Expanding CoGroupByKey operations into optimizable parts.
May 03, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.768Z: Expanding GroupByKey operations into optimizable parts.
May 03, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.808Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.869Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.893Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.924Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:33.956Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:34.424Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:45:34.511Z: Starting 5 workers in us-central1-f...
May 03, 2020 12:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-03T00:46:03.341Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2020 12:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:04.886Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 03, 2020 12:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:04.914Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 03, 2020 12:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:10.242Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 03, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:28.776Z: Workers have started successfully.
May 03, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:28.807Z: Workers have started successfully.
May 03, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:46:59.944Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 03, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:47:00.123Z: Cleaning up.
May 03, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:47:00.194Z: Stopping worker pool...
May 03, 2020 12:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:48:41.821Z: Autoscaling: Resized worker pool from 5 to 0.
May 03, 2020 12:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-03T00:48:41.867Z: Worker pool stopped.
May 03, 2020 12:48:48 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-02_17_45_26-1130745998024889789 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 91aedcaf-6803-42cd-bb7b-8f3c157bcb40 and timestamp: 2020-05-03T00:48:48.657000000Z:
Metric: Value:
read_time 14.95
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 42.074 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/3kfft35maoeja
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #455
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/455/display/redirect>
Changes:
------------------------------------------
[...truncated 285.51 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 02, 2020 6:48:15 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 6:48:15 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 02, 2020 6:48:15 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 6:48:15 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 02, 2020 6:48:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 02, 2020 6:48:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 02, 2020 6:48:18 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 02, 2020 6:48:19 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 02, 2020 6:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 02, 2020 6:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-YXVzYKfhELLAouejb5OSNQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-g6Z04R_j9n_UevfouZdm3w.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-_7Ydp9of9GJV7b5yWoRsZA.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-dPxRG5F-kOWxmw-oDnyDAQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-aEYieBa9oGr-RVjQSiY6AQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-mwscnfCtWzJQxVaYI654gA.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-JiG4kEc6R3oufS6ekATMfQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-fJ904eGEGs8I8x3r9kFwlQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-NpE5joSGps2vu3QwuG0ekQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-8Fa8AoZoeaABEkcsjFq-fw.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-GOCOMrr14WuRdliE7wExEg.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-TRd-GQdrFh1GJfNVkfDbwQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-cIv9kcYTdHWLLPZ7WRa3vg.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-YQJt3f_W7NxXAhPQSOxGwg.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-z_piq02d8CbUf6pjAEM63A.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nEXG9wca1Pm_TNjZiTHR5Q.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-sbuMoZHLQuXDW0VQxITdRA.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-NpNrzAo8FRWxJzHiRgkdEA.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-3YQ8xQqtL-ti9ro73L6w6w.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-PqnUkTL_L2hPgr1akrU8Yw.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-jcQ9OjgYT38Dxl0Xt8H1WQ.jar
May 02, 2020 6:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nEXG9wca1Pm_TNjZiTHR5Q.jar
May 02, 2020 6:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-kjOevFyE3LOzFg7sP1l4SA.jar
May 02, 2020 6:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-4oBxIvfTuD20OpSaD44tvw.jar
May 02, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-CilnFVrmHcApfHIlAI38YQ.jar
May 02, 2020 6:48:25 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nEXG9wca1Pm_TNjZiTHR5Q.jar&uploadType=resumable&upload_id=AAANsUlpqmw_OubUHXl4EWj19XkdiKdlW_6yuaMqasdlaar4NN8zmMhPURLOclwOe8ZAkZ2Ax5CuhCWyJxvkpyET7HQsTBipLw.
May 02, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Y21W5IdZGs-hV7nXEr6Hww.jar
May 02, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 02, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-AoIIA2YNQhpibqESpFazlw.jar
May 02, 2020 6:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-vfpI-J9ScSQHUErCdXLaiw.jar
May 02, 2020 6:48:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nEXG9wca1Pm_TNjZiTHR5Q.jar
May 02, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 12 seconds
May 02, 2020 6:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 02, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 02, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 02, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 02, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 02, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash OSAsCRONrmvWz24XMOK2CA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-OSAsCRONrmvWz24XMOK2CA.pb
May 02, 2020 6:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 02, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-02_11_48_35-8976129892279711837?project=apache-beam-testing
May 02, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-02_11_48_35-8976129892279711837
May 02, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-02_11_48_35-8976129892279711837
May 02, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T18:48:35.316Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 02, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:38.827Z: Checking permissions granted to controller Service Account.
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:42.342Z: Worker configuration: n1-standard-1 in us-central1-f.
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:42.965Z: Expanding CoGroupByKey operations into optimizable parts.
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.016Z: Expanding GroupByKey operations into optimizable parts.
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.061Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.143Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.165Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 02, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.195Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 02, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.232Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 02, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.580Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:48:43.664Z: Starting 5 workers in us-central1-f...
May 02, 2020 6:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T18:49:03.408Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 02, 2020 6:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:49:09.889Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 02, 2020 6:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:49:09.930Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 02, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:49:15.367Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 02, 2020 6:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:49:30.393Z: Workers have started successfully.
May 02, 2020 6:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:49:30.436Z: Workers have started successfully.
May 02, 2020 6:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:50:06.192Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 6:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:50:06.401Z: Cleaning up.
May 02, 2020 6:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:50:06.482Z: Stopping worker pool...
May 02, 2020 6:51:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:51:34.054Z: Autoscaling: Resized worker pool from 5 to 0.
May 02, 2020 6:51:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T18:51:34.094Z: Worker pool stopped.
May 02, 2020 6:51:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-02_11_48_35-8976129892279711837 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0c042f12-e7a6-490b-9efe-b2f8fccebdf2 and timestamp: 2020-05-02T18:51:42.120000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.527
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.072 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.084 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 48.902 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 54s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/uyrcfivd7ip2i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #454
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/454/display/redirect>
Changes:
------------------------------------------
[...truncated 285.03 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 02, 2020 12:56:05 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 12:56:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 02, 2020 12:56:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 12:56:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 02, 2020 12:56:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 02, 2020 12:56:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 02, 2020 12:56:06 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 02, 2020 12:56:07 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 02, 2020 12:56:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 02, 2020 12:56:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-rVoP-cAuob0K4PONYbBPag.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-CCLofG7lM49eA36BtE778Q.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-CSxnP_e0yZcOEa89zQE_vQ.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-e7WUfpn7tSP55SdmnwKjVA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-W7JOuK5U2AutMLnMz4Lt3w.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-JoeG-VX1ge1ofykKKdlwnw.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-1zq08IiKIe68RSx-J3yNEA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-JFPoLXEMuerQIy1S0yg4Ug.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-ZTNiqd5Iu1O7dsq7rIe0CQ.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-2tsGKkkZDgkvbP5XXT1XzA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-QavcIo49_KS_BOr6E61xpg.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-GgP8sjsqkiOHFtnhUImetg.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-vAlbLwd9tdZun38uR0Lsxw.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-BJgLHoM4CAuOIykalIHLHg.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-7pkHoItez7G4I_ZEh3sKMA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-eIpHE4b0yL5WDHaWNpgFSg.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-cm2OopZPzGIYuuC3fMO3LA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-4kn7MB9cFLvr-7BrKoYS2A.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-b1TOZTcRXJMY20YWEoHOJw.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-OAxtA8kC5JJvqDNVAxFI7A.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-IJ-5F7J5xh_nu4Dm43kAQQ.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-y-kO1prcV4o6ThnZ0n9hAg.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-bO7lwzBrl5OJ4e4euXqMEA.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-pjhE5EiT3-fMa3qH2E3r7Q.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cSnhS4iNW1uw8wUfTOVkHQ.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Vh_VOX1SCsWYMmhGbTEH-A.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cSnhS4iNW1uw8wUfTOVkHQ.jar
May 02, 2020 12:56:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-XtLjHmQQdN7Y1bcMGlPoAw.jar
May 02, 2020 12:56:09 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cSnhS4iNW1uw8wUfTOVkHQ.jar&uploadType=resumable&upload_id=AAANsUnAEGVzIFl2oBvs0ar8nsjXTzR1Z6U3HcF-lxYLHxwPq_RT7ZDnNtqBU2_pwa61vd4YkEqNsmvUcVuBRwTPtfc.
May 02, 2020 12:56:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 02, 2020 12:56:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cSnhS4iNW1uw8wUfTOVkHQ.jar
May 02, 2020 12:56:14 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 02, 2020 12:56:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 6HDOSzuDzN58UI76sCZNOQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6HDOSzuDzN58UI76sCZNOQ.pb
May 02, 2020 12:56:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 02, 2020 12:56:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-02_05_56_16-16974268272717836247?project=apache-beam-testing
May 02, 2020 12:56:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-02_05_56_16-16974268272717836247
May 02, 2020 12:56:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-02_05_56_16-16974268272717836247
May 02, 2020 12:56:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T12:56:16.091Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 02, 2020 12:56:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:18.949Z: Checking permissions granted to controller Service Account.
May 02, 2020 12:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:22.947Z: Worker configuration: n1-standard-1 in us-central1-f.
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.555Z: Expanding CoGroupByKey operations into optimizable parts.
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.594Z: Expanding GroupByKey operations into optimizable parts.
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.617Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.700Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.736Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.771Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:23.808Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:24.209Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 12:56:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:24.285Z: Starting 5 workers in us-central1-f...
May 02, 2020 12:56:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T12:56:51.185Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 02, 2020 12:56:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:51.832Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
May 02, 2020 12:56:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:51.863Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
May 02, 2020 12:56:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:56:57.250Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 02, 2020 12:57:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:57:10.323Z: Workers have started successfully.
May 02, 2020 12:57:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:57:10.362Z: Workers have started successfully.
May 02, 2020 12:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:57:42.600Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 12:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:57:42.811Z: Cleaning up.
May 02, 2020 12:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:57:42.887Z: Stopping worker pool...
May 02, 2020 12:59:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:59:17.085Z: Autoscaling: Resized worker pool from 5 to 0.
May 02, 2020 12:59:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T12:59:17.123Z: Worker pool stopped.
May 02, 2020 12:59:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-02_05_56_16-16974268272717836247 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 43dce6fa-b91b-4037-875e-70a9ec0f0a59 and timestamp: 2020-05-02T12:59:23.509000000Z:
Metric: Value:
read_time 13.592
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 25.075 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/qfjod5plq6kay
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #453
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/453/display/redirect>
Changes:
------------------------------------------
[...truncated 281.71 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 02, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 02, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 02, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 02, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 02, 2020 6:45:30 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 02, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 02, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 02, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6fW7j0H7gTXIsMtUXTQVPg.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-uZ4AaR5OvckeRr6eH3l2Rg.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-XAJWQC8jhKPe7sGrApOezw.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-7P845dcbLTAw74WTwWDo4A.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-66OozSpDFEI1QJZ34gZ0yA.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-sq3ZGh8r2Uoig5qaL4VpSQ.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-XCqBiB9PwOylIDtqmmfzjA.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-XTwCvWvohHn5EF0G6slDYA.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-VsEAo9Ksl1cqJLdpcZvnQA.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-rkF9buLvHOlGGGR8nmh4Pg.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-dlESAEm-Lds3_Sq1YZCSaQ.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-tbQFQ23L_mJS7Ym7UlkX9Q.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-IqPy7-S0fT2ddkZcm4JF5w.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-4TonwcyH92jveOU6hmxvJA.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-M-_J0i5ABxt3cvdh2vQT3Q.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-9taHwJ_or1cyfSvpE6L_Dw.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-S43D6VZHOi54MlFs3lxpqQ.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-v05Lk82rmRJSZc2zLbxg9A.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-qDtzv6uglO7po7HWExaLfQ.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-5cZ32gxWYYyaJK3P0vyKOg.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-B9UPBu-LMwxCU2lFOHsXGw.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-lFb5Rw8NZ30lgc2tTbom3w.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-wNsPwXH78J5RbcGCUbt0Kg.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-DCG6_M6has-no5mM4XwnFw.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-d1iBRkKzD_u18WCeOSLMpw.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ze7oFNPq-MtFv8dX-n5o7g.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ze7oFNPq-MtFv8dX-n5o7g.jar
May 02, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-fKmo5vsKBoh-oTlf6CMlTw.jar
May 02, 2020 6:45:33 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ze7oFNPq-MtFv8dX-n5o7g.jar&uploadType=resumable&upload_id=AAANsUmdidnNan2QbO5XPxRd5QMKErBIheE2zDqO0vYUsiwfJAirf2hK2LRFPjEB7HKQrh7Wi0C0FIfPQ9r31pj_Wzo.
May 02, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 02, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ze7oFNPq-MtFv8dX-n5o7g.jar
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 02, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 8AgAwYm3aCPZEYbBHDe8OQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8AgAwYm3aCPZEYbBHDe8OQ.pb
May 02, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 02, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-01_23_45_38-1277837789498749852?project=apache-beam-testing
May 02, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-01_23_45_38-1277837789498749852
May 02, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-01_23_45_38-1277837789498749852
May 02, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T06:45:38.598Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 02, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:41.860Z: Checking permissions granted to controller Service Account.
May 02, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:46.235Z: Worker configuration: n1-standard-1 in us-central1-f.
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:46.808Z: Expanding CoGroupByKey operations into optimizable parts.
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:46.858Z: Expanding GroupByKey operations into optimizable parts.
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:46.901Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.062Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.102Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.141Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.177Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.753Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:45:47.933Z: Starting 5 workers in us-central1-f...
May 02, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T06:46:03.035Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 02, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:46:16.466Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 02, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:46:36.377Z: Workers have started successfully.
May 02, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:46:36.417Z: Workers have started successfully.
May 02, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:47:12.378Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:47:12.609Z: Cleaning up.
May 02, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:47:12.718Z: Stopping worker pool...
May 02, 2020 6:48:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:48:52.481Z: Autoscaling: Resized worker pool from 5 to 0.
May 02, 2020 6:48:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T06:48:52.538Z: Worker pool stopped.
May 02, 2020 6:48:58 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-01_23_45_38-1277837789498749852 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 216ed79a-5105-4083-95c2-c940afebbfa1 and timestamp: 2020-05-02T06:48:58.907000000Z:
Metric: Value:
read_time 17.431
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 36.806 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/b6hwke6mc57cu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #452
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/452/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9600] Get metrics in Flink uber jar job server.
[kcweaver] [BEAM-9600] Add a test for Flink REST metrics.
[rohde.samuel] [BEAM-9692] Replace the apply_read with a PTransformOverride
[bhulette] remove PipelineRunner.fromOptions call
[mxm] [BEAM-2128] Remove dependency on PipelineRunner#fromOptions to stage
[bhulette] Modify PipelineTest to check if exception is thrown on Pipeline run
[bhulette] Call FileSystems.setDefaultPipelineOptions in
[rohde.samuel] move redundant testing pieces to function
[mxm] [BEAM-9801] Pass in fire timestamp and pane info to timer callback
[mxm] [BEAM-9733] Repeatedly fire in batch mode until there are no more timers
[kcweaver] [BEAM-9860] Require job_endpoint when using PortableRunner.
[kcweaver] Replace portableWordCount tests with portableWordCountFlinkRunner.
[robertwb] Test simplification.
------------------------------------------
[...truncated 286.37 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:538)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 02, 2020 12:46:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 02, 2020 12:46:14 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 02, 2020 12:46:14 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-B50P_T7Bli56F0eoLWl3Ig.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Cp0BOj4gfd0ZT4O2qRmqrw.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-vgO-YPzStpjfgI3Tf6vIRg.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-XagqWxixI4x-GLBppJE14g.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-i6LXDN39WMpRN-A5i8kjkQ.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-yY3qn9nJUPmFgySmQ6Yhow.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-jhcPBK7xFqfFVN1Yg2uuLg.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-s6bZZmtKTmH27N_iu90QVg.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-uzNRFJpIxT8mGhqWosP7YQ.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-JUeMkEmtrcD0pZ2bMystDA.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-xtMya2pEUpMElfFHXH6noA.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-kj6NDCFRqTs71Ryi4w0b3Q.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-utjGz8Y6wGQEymfRm6FEyA.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-QeAYv2u-Xoxn-tQvpZvRYg.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-PaXziZuZ0_MLekFIkU8gQw.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-0egW6d1H3pI7A_9ZRFkguQ.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-NT3K8AD01lEosSZETGQaOQ.jar
May 02, 2020 12:46:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-_c9F4EXRoZ-amy3KgehtmA.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-JlBiTp8kEXT7xAAIMwxQVg.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-jtyLIcE5PDI6xeg8CyeEZA.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-j_6zaAzd9gZ35pmQPWg48Q.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-OmP7DSPLgBe4ampP3x0Kbw.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests--Kiua6Vq0aElG5otOXHBoQ.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-tpejKQNAoty0cD5MiRxuAA.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9pKbLIZdAh7aUql6ddGeXA.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9pKbLIZdAh7aUql6ddGeXA.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-xU3LtdjQUe6NzPCipIdQBw.jar
May 02, 2020 12:46:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-vKa5eZbkK_p1Vc9pSSXEFg.jar
May 02, 2020 12:46:17 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9pKbLIZdAh7aUql6ddGeXA.jar&uploadType=resumable&upload_id=AAANsUkppg-VY0bY7Vnzbu77NRyl5o1Jy-S67jt4Bih20MLfV8KPDcLB-Lb5ErdxCufxm5Ueekad-QYrfFnwBL4qo38.
May 02, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 02, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9pKbLIZdAh7aUql6ddGeXA.jar
May 02, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 02, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash 1zRd-DpaB2_dWLp8175t6A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1zRd-DpaB2_dWLp8175t6A.pb
May 02, 2020 12:46:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 02, 2020 12:46:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-01_17_46_23-2581635633307054460?project=apache-beam-testing
May 02, 2020 12:46:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-01_17_46_23-2581635633307054460
May 02, 2020 12:46:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-01_17_46_23-2581635633307054460
May 02, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T00:46:23.269Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 02, 2020 12:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:26.751Z: Checking permissions granted to controller Service Account.
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:30.484Z: Worker configuration: n1-standard-1 in us-central1-f.
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.236Z: Expanding CoGroupByKey operations into optimizable parts.
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.279Z: Expanding GroupByKey operations into optimizable parts.
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.314Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.394Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.436Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.469Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.505Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:31.972Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 12:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:32.053Z: Starting 5 workers in us-central1-f...
May 02, 2020 12:46:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-02T00:46:40.070Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 02, 2020 12:46:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:57.276Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 02, 2020 12:46:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:46:57.309Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 02, 2020 12:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:02.783Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 02, 2020 12:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:15.150Z: Workers have started successfully.
May 02, 2020 12:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:15.188Z: Workers have started successfully.
May 02, 2020 12:47:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:48.263Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 02, 2020 12:47:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:48.485Z: Cleaning up.
May 02, 2020 12:47:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:47:48.580Z: Stopping worker pool...
May 02, 2020 12:49:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:49:24.649Z: Autoscaling: Resized worker pool from 5 to 0.
May 02, 2020 12:49:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-02T00:49:24.695Z: Worker pool stopped.
May 02, 2020 12:49:31 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-01_17_46_23-2581635633307054460 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 94e40630-de9d-43bb-8a0b-ee3c5cf4d921 and timestamp: 2020-05-02T00:49:31.403000000Z:
Metric: Value:
read_time 13.23
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 25.883 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/aduimgsgb7lxo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #451
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/451/display/redirect?page=changes>
Changes:
[robertwb] Guard grpc import.
------------------------------------------
[...truncated 283.30 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 01, 2020 7:02:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 01, 2020 7:02:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 01, 2020 7:02:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-5xJaUL3TxH_Gyx06D5nODA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-gO3TLOFMtcIHXfc7YWZVug.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-pK5t5xIGFjgRuBX_BAL2kw.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-OSu7-BlsraWLnf9eZhq1aQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-pWzhoznA7XFhk8Q5EP8VFw.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-gx9l1uYEimF8tpsY2QIMLw.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-4OHrbgxewdMxt7tq-cApjw.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tEzuMV8wu1DmVWR9JHeQUA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ntO59TEsP0j0PLhOW9Fe4g.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-nt44-oYZG33hKPV07hcsOA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-KK7q1sh2r7-Yb8GII7Ud_A.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-sJB8CNK2qiUQGPXmb9GwRw.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-J2nnev53YoV3jMkAug2jdQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-M8V-97NvZbTJjpqJWtcGTA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-JOpxqv0MaowixvUAkyhl8Q.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-hI4_hKnkjwTY7LpBr6Xg5w.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ITij0QjuhMANonzXeXDkZA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-_4MWAyu0MFOCWnchUdoP_Q.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ZryfGJyl412LlNJXrre-XQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-MUxKBp-vl6SkRPZn_uHi9g.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-eJERZF-WtUVjuNLzA10OvQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-aSaLahRtgS5uyY2z9o99-A.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Nchq3zgHz_r4DnaFLXNpHA.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-CEp-wsQoinRKMHfPeMu0HQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--DyVoAIWKb-hhIDIWeKnJQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-4cWAPB_ISCjKh7yIvmOmNQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--DyVoAIWKb-hhIDIWeKnJQ.jar
May 01, 2020 7:02:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-nl5lkUOoaowmqcf_i2iF2w.jar
May 01, 2020 7:02:37 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--DyVoAIWKb-hhIDIWeKnJQ.jar&uploadType=resumable&upload_id=AAANsUkdzYHX6e9P_ps3ZZW0p74zt5I1vKnkNc4AhypRUb-pWZG6UsdJWNjJsUbfR6eeUfLu2dgj-4zaOChd0x-t-JU.
May 01, 2020 7:02:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 01, 2020 7:02:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--DyVoAIWKb-hhIDIWeKnJQ.jar
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 01, 2020 7:02:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash JLOVigLvW7hB4Jb5gi8Yvg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-JLOVigLvW7hB4Jb5gi8Yvg.pb
May 01, 2020 7:02:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 01, 2020 7:02:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-01_12_02_42-11476058222936969833?project=apache-beam-testing
May 01, 2020 7:02:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-01_12_02_42-11476058222936969833
May 01, 2020 7:02:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-01_12_02_42-11476058222936969833
May 01, 2020 7:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T19:02:42.204Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 01, 2020 7:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:45.806Z: Checking permissions granted to controller Service Account.
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:49.606Z: Worker configuration: n1-standard-1 in us-central1-f.
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.413Z: Expanding CoGroupByKey operations into optimizable parts.
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.452Z: Expanding GroupByKey operations into optimizable parts.
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.484Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.568Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.601Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.634Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 01, 2020 7:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:50.668Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 01, 2020 7:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:51.036Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 7:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:02:51.118Z: Starting 5 workers in us-central1-f...
May 01, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T19:03:04.899Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 01, 2020 7:03:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:03:19.253Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 01, 2020 7:03:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:03:38.873Z: Workers have started successfully.
May 01, 2020 7:03:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:03:38.916Z: Workers have started successfully.
May 01, 2020 7:04:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:04:12.576Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 7:04:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:04:12.805Z: Cleaning up.
May 01, 2020 7:04:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:04:12.900Z: Stopping worker pool...
May 01, 2020 7:06:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:06:17.768Z: Autoscaling: Resized worker pool from 5 to 0.
May 01, 2020 7:06:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T19:06:17.819Z: Worker pool stopped.
May 01, 2020 7:06:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-01_12_02_42-11476058222936969833 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e79c7a2f-36b9-45fd-918b-55d02e55648c and timestamp: 2020-05-01T19:06:25.441000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.288
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 58.97 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 2s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/hp7jsjusotq2a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #450
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/450/display/redirect>
Changes:
------------------------------------------
[...truncated 283.79 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 01, 2020 1:06:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 1:06:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 01, 2020 1:06:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 1:06:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 01, 2020 1:06:05 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 01, 2020 1:06:05 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Dz7UCIuZQi6LkY7NSJKHRQ.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-AzaSK4It2U3MyK9L3R73cQ.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Hwvx_pVtxJS6RNs8CSv5Rw.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-0kaHKTiqU61tIgtSPX9jUQ.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-DFqQ-5_tO1taXChVaMLWOg.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Q_8UaxwmHilsKuGfEwQnVA.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-CQD2yf1G8DAWi2T5d4vQ7Q.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-SiZhu1CpxmO9kY8AN7LV9g.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Gw1NuaGNen1oNTXDgYxVHw.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-9toldQxG8tkHX0GLQbBFHg.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-VkIjRkZ1rt7TVqGOVU7PEg.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-9YNJ54KDZNshuQ7WhKTzpg.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Sej7C2cBx4MHD3VvBX5iDw.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-GNsH9-AvLi4Aued4Ft-Fjg.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-IZGFR4I8JAkoAJ4e4Ji6jw.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-_xr1Q3HEXrA7TKazceTJeQ.jar
May 01, 2020 1:06:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-JAZIZAd4u8at4YlWem_Cgw.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-FOliCuVgPKapdAyMabC83A.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-FapjXKjbZfb1-AOAZ0I2Lw.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-_QARglzkzgrbXi3edS3DCA.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-jewYkLJQtd-zGdufN7dxiw.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-mZgcYr7BbMyjMXrXuMJHnQ.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-FjFCQV0ni11DcOx37wmjLw.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-mVAhHs0ne1lNJNeG4DP-yw.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-3g8oVEu4LDIeuDmoyGJkEA.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-RP3nHSeO6BAhGQinCT53cQ.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-RP3nHSeO6BAhGQinCT53cQ.jar
May 01, 2020 1:06:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-DNU5ioBP44MDXczl9CxSTQ.jar
May 01, 2020 1:06:08 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-RP3nHSeO6BAhGQinCT53cQ.jar&uploadType=resumable&upload_id=AAANsUkDaSfcL5lFGT4tmxs9u9Ne8P0vchbBUxV2T63CkxSQnZDWL-JHEPhtQAs6qI4ureMIsEvrVa8AjpoIRvh3wG8.
May 01, 2020 1:06:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 01, 2020 1:06:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-RP3nHSeO6BAhGQinCT53cQ.jar
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 01, 2020 1:06:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 01, 2020 1:06:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash FAsyHzJmpebjn9_rvtnB7g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-FAsyHzJmpebjn9_rvtnB7g.pb
May 01, 2020 1:06:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 01, 2020 1:06:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-05-01_06_06_17-17622334419479134020?project=apache-beam-testing
May 01, 2020 1:06:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-05-01_06_06_17-17622334419479134020
May 01, 2020 1:06:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-05-01_06_06_17-17622334419479134020
May 01, 2020 1:06:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T13:06:17.511Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 01, 2020 1:06:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:20.739Z: Checking permissions granted to controller Service Account.
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:24.220Z: Worker configuration: n1-standard-1 in us-central1-f.
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:24.857Z: Expanding CoGroupByKey operations into optimizable parts.
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:24.898Z: Expanding GroupByKey operations into optimizable parts.
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:24.940Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.055Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.097Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.132Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.169Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.619Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 1:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:25.692Z: Starting 5 workers in us-central1-f...
May 01, 2020 1:06:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T13:06:53.116Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 01, 2020 1:06:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:56.801Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 01, 2020 1:06:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:06:56.837Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
May 01, 2020 1:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:02.331Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 01, 2020 1:07:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:21.027Z: Workers have started successfully.
May 01, 2020 1:07:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:21.061Z: Workers have started successfully.
May 01, 2020 1:07:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:55.907Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 1:07:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:56.107Z: Cleaning up.
May 01, 2020 1:07:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:07:56.199Z: Stopping worker pool...
May 01, 2020 1:09:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:09:47.442Z: Autoscaling: Resized worker pool from 5 to 0.
May 01, 2020 1:09:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T13:09:47.487Z: Worker pool stopped.
May 01, 2020 1:09:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-01_06_06_17-17622334419479134020 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0a7df3a7-aba7-45ce-b283-98d6beefa24e and timestamp: 2020-05-01T13:09:52.469000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.679
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 57.091 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/qnjatkgdk2tsi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #449
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/449/display/redirect>
Changes:
------------------------------------------
[...truncated 284.66 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 01, 2020 6:45:47 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-lPtfgta0fY85PqdW6S8x9A.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-h5F8JbfDF9OXjlVjIPrHKg.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tQNS96kswsuSd2-OxZ0Xvg.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-a3ULT8D2Lo8ivkaJtQfCVg.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-jSPGeYPkGrQEmEjA-lNL6A.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-mJWZIHTsMoYwGtTqvDFkpQ.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-fnS5I4VUX8JhLNTQVpvHKQ.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-MktZNFAuYhqYu-2CvMk1vQ.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-J48KGTkMMOa_Y812pawFRw.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-vX546t1nVlCQXZhptR2cAA.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-5BCWzVoivb-qKzzyw843fA.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-_F8gYcil1sTOEdFi8iREyg.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-qrvXq2DC_AkHR8P5QaBhyQ.jar
May 01, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-WnFWfRCiGFe-7SVdzPkyqg.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-YJf4nD-LHCpjewH6r0LVnA.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-XCdeu1u74gaP71vyAcKtsg.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-GS28avZ-I1wSxvRSLm8hqg.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-oswwENu0JBRz90mdExIUew.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-RdBe5hriWnWv0ulEyyJYRA.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-6o1qtiVcaCpMetEGxDzlYg.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-bcifCq7RNl1DpoByeJg6aA.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-zQ92MYAr725f7onSHhQ59A.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-7YynMNUeLOpihwEUu9_Yiw.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-IyyMh0rRlO_ekP5iP49JBw.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-NVHEA-xRIr6ZoLMONO19mg.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-HIllJLC53cgHbR2nDgZWwA.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9G1k9HPp6bdlBzHvFmcrcQ.jar
May 01, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9G1k9HPp6bdlBzHvFmcrcQ.jar
May 01, 2020 6:45:50 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9G1k9HPp6bdlBzHvFmcrcQ.jar&uploadType=resumable&upload_id=AAANsUnzUx1-oxNmKHc6ow0wBU2wipFvFEKJRtktPZt-YebXoTRuwCzSz2XPwXEmoDPlfv_y8Xsq68qX8F3RO2yAwCw.
May 01, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 01, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-9G1k9HPp6bdlBzHvFmcrcQ.jar
May 01, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
May 01, 2020 6:45:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 3vU7-lMQLho5FXGIkKJ1GQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3vU7-lMQLho5FXGIkKJ1GQ.pb
May 01, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 01, 2020 6:45:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-30_23_45_58-6764038853434485646?project=apache-beam-testing
May 01, 2020 6:45:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-30_23_45_58-6764038853434485646
May 01, 2020 6:45:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-30_23_45_58-6764038853434485646
May 01, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T06:45:58.965Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 01, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:03.669Z: Checking permissions granted to controller Service Account.
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:10.131Z: Worker configuration: n1-standard-1 in us-central1-f.
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:10.780Z: Expanding CoGroupByKey operations into optimizable parts.
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:10.836Z: Expanding GroupByKey operations into optimizable parts.
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:10.910Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:11.115Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 01, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:11.183Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 01, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:11.234Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 01, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:11.286Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 01, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:11.942Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:12.110Z: Starting 5 workers in us-central1-f...
May 01, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T06:46:21.920Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 01, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:39.476Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
May 01, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:39.547Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
May 01, 2020 6:46:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:44.995Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 01, 2020 6:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:59.168Z: Workers have started successfully.
May 01, 2020 6:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:46:59.214Z: Workers have started successfully.
May 01, 2020 6:47:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:47:34.297Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 6:47:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:47:34.563Z: Cleaning up.
May 01, 2020 6:47:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:47:34.762Z: Stopping worker pool...
May 01, 2020 6:49:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:49:09.534Z: Autoscaling: Resized worker pool from 5 to 0.
May 01, 2020 6:49:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T06:49:09.627Z: Worker pool stopped.
May 01, 2020 6:49:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-30_23_45_58-6764038853434485646 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3c74bcc7-68be-4306-ae2c-cd7b6439dfd8 and timestamp: 2020-05-01T06:49:15.176000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.508
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 36.652 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/hkddm3bceto7e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #448
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/448/display/redirect?page=changes>
Changes:
[sniemitz] [BEAM-9795] Support custom avro DatumWriters when writing to BigQuery
[github] skip pulling licenses by default (#11548)
[boyuanz] Update container version
[github] [BEAM-8078] streaming_wordcount_debugging.py is missing a test (#11568)
[github] Merge pull request #11538 from [BEAM-9831] Improve UX for HL7v2IO
------------------------------------------
[...truncated 282.69 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
May 01, 2020 1:01:53 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
May 01, 2020 1:01:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 01, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-E-o-3x26-9GxelQLiih31g.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-v0aM4xNimpnNBZe0BP63eg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-ZV6HVv7rVt0MBjcjuGxHdg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Hn4OW2sxztkwKeuZ4i9TJw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-vrGQ8O0kkLJuhQq2DLYK3A.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-QBYUsYfHCS7fkIluSJy9aw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-wUcDDlBl69EK8T8_vxuing.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Xd7WSPl1xF-LHojLXdnKpg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-y7sQ88Aodwz0cvtn5LyyBA.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-oxK90vPmCrWG58193zKNcw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-aaVjtd1_Quh7SzImsAjm5w.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-6uy7zdfF3T_vWerj1xAHbg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-uipSWsoPCXpOnTSijl0ekA.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-eMIe3f_kIpRT9TkCiywLIg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-okWqa6WYlFwXoW_nGpDKbQ.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3QBJ8dIe7uiU2-17X1z32A.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Kjaat8Bf0CEpspv0WVaqIg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-82VI3Wj5nLh-Yw9v6vTfqw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-OK00mpkfmw2Ypsq5yGQgPg.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-22jSMafOVV9cqDZtkDm34g.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-b4uHRDmBr4PFWF3FJ-wDzw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-G7bnm7eQkq0RlAjYINUOOQ.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-RUjhlAyHeU5f4kU6acFdtQ.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-bTOhzKemdesczU-MnY28TQ.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vdjRQ7h0mHihoWr2VaBGKA.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-nYHqnCYKOzsNm6aYGujdtw.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vdjRQ7h0mHihoWr2VaBGKA.jar
May 01, 2020 1:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JOzzll_mgQXHjkly1msxUQ.jar
May 01, 2020 1:01:56 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vdjRQ7h0mHihoWr2VaBGKA.jar&uploadType=resumable&upload_id=AAANsUkbTHltGqenEPciOvgl5lyKQomuCc_d4sZXr9Iu1KoaVOZiSElNBNxRPtJ9fWy6ENnyUWifHzUJOL5lIjrJpNI.
May 01, 2020 1:01:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
May 01, 2020 1:01:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vdjRQ7h0mHihoWr2VaBGKA.jar
May 01, 2020 1:01:59 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
May 01, 2020 1:02:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash LlvYkPf-SHX8oIjlN2zmaQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LlvYkPf-SHX8oIjlN2zmaQ.pb
May 01, 2020 1:02:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
May 01, 2020 1:02:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-30_18_02_01-10067151511895160918?project=apache-beam-testing
May 01, 2020 1:02:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-30_18_02_01-10067151511895160918
May 01, 2020 1:02:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-30_18_02_01-10067151511895160918
May 01, 2020 1:02:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T01:02:01.238Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
May 01, 2020 1:02:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:05.294Z: Checking permissions granted to controller Service Account.
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:10.520Z: Worker configuration: n1-standard-1 in us-central1-f.
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.236Z: Expanding CoGroupByKey operations into optimizable parts.
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.339Z: Expanding GroupByKey operations into optimizable parts.
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.378Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.468Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.503Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.544Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.580Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:11.993Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 1:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:12.069Z: Starting 5 workers in us-central1-f...
May 01, 2020 1:02:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-01T01:02:31.934Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 01, 2020 1:02:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:39.394Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
May 01, 2020 1:02:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:39.423Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
May 01, 2020 1:02:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:02:45.043Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
May 01, 2020 1:03:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:03:02.507Z: Workers have started successfully.
May 01, 2020 1:03:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:03:02.542Z: Workers have started successfully.
May 01, 2020 1:03:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:03:33.738Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
May 01, 2020 1:03:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:03:33.951Z: Cleaning up.
May 01, 2020 1:03:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:03:34.031Z: Stopping worker pool...
May 01, 2020 1:05:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:05:33.283Z: Autoscaling: Resized worker pool from 5 to 0.
May 01, 2020 1:05:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-01T01:05:33.334Z: Worker pool stopped.
May 01, 2020 1:05:39 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-30_18_02_01-10067151511895160918 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 921c91eb-e454-4130-8f39-05f0ce842692 and timestamp: 2020-05-01T01:05:39.413000000Z:
Metric: Value:
read_time 13.268
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 54.678 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/3ikvpmiyue2ue
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #447
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/447/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5192] move metadata serialization configuration code to
[github] [BEAM-9720] Add Aws Http client configurations capability. (#11341)
------------------------------------------
[...truncated 282.35 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 30, 2020 6:58:48 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 6:58:48 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 30, 2020 6:58:48 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 6:58:48 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 30, 2020 6:58:49 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 30, 2020 6:58:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 30, 2020 6:58:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 30, 2020 6:58:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-aCme-uOwBbG24ngfLaWpGw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-60api0ZeOQhzQKs6leydCQ.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6dHYvtfdBue7lCNBdDsf6Q.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-NnR9j0MDupJXPWSY4QlP-w.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-hMG0xVnJpicWL8CcFzjzlg.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-TDPObQJDZjvlGlzYmZscEw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-uYcVh6ZrvpK4e-mPgNaLPQ.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-1vqdJQgGZg5IGo4vaLpQUw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-EX3wv7SkbpEGZVD-PchKCg.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-3MhVrfDOQhnXAR65-Imbsg.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-r-pWpRwG780w9VLo0cozjw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-vEvDnydaoWLpoP9i3dfW1g.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-foWOQhXJZxPVe9z6jvITpA.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-6RQLGV9XO8K-KpYxF773HQ.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-R3-PdePaQ134rbuTWUt_kw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-EfTM0WOZaOE9ZOhmMYmGjw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-BH-8vH8rtVJHxHdxZGamfw.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-GTZaKfhmEDOAOmUJ7g9gyQ.jar
Apr 30, 2020 6:58:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-ICNfAgljWZ9haYIIFI-VLg.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-RRUQYXnq9vEIJSWcnfJq_w.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3zyuJ97yG2S94eq1irv3og.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-IoZoWnLLVDjavLOb3HMMJQ.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-43DwYlRzv7VeoPYwIPS5Vg.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-AnW00uVhzC5mKyKqTlFUUw.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-N-b3jykLzAPLiCC-k_LdYw.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-GX3yhBW2JcrZtVGRMpVyyQ.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-N-b3jykLzAPLiCC-k_LdYw.jar
Apr 30, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-xSSuf3jOrvkQnFGkkjIutQ.jar
Apr 30, 2020 6:58:52 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-N-b3jykLzAPLiCC-k_LdYw.jar&uploadType=resumable&upload_id=AAANsUncRqUm-Xmu0z3wL_2DrucmSI6iny70ZGWN5SrSSbzvFXGrJM-qZ0yHQwyQppBhYFlaRpdBMlwGi0MOJ0Ay-wW_Ktu7fw.
Apr 30, 2020 6:58:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 30, 2020 6:58:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-N-b3jykLzAPLiCC-k_LdYw.jar
Apr 30, 2020 6:58:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 30, 2020 6:58:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash TIimTU9W1x3Ym_RhnsI75A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-TIimTU9W1x3Ym_RhnsI75A.pb
Apr 30, 2020 6:59:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 30, 2020 6:59:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-30_11_59_00-16458478598023341465?project=apache-beam-testing
Apr 30, 2020 6:59:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-30_11_59_00-16458478598023341465
Apr 30, 2020 6:59:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-30_11_59_00-16458478598023341465
Apr 30, 2020 6:59:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T18:59:00.097Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 30, 2020 6:59:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:03.068Z: Checking permissions granted to controller Service Account.
Apr 30, 2020 6:59:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:07.315Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 30, 2020 6:59:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:07.929Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 30, 2020 6:59:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:07.968Z: Expanding GroupByKey operations into optimizable parts.
Apr 30, 2020 6:59:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:07.995Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.081Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.108Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.132Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.171Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.562Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 6:59:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:08.646Z: Starting 5 workers in us-central1-f...
Apr 30, 2020 6:59:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T18:59:32.750Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2020 6:59:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:35.184Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 6:59:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:35.225Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 30, 2020 6:59:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:40.625Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 7:00:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:59.905Z: Workers have started successfully.
Apr 30, 2020 7:00:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T18:59:59.937Z: Workers have started successfully.
Apr 30, 2020 7:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T19:00:33.572Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 7:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T19:00:33.758Z: Cleaning up.
Apr 30, 2020 7:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T19:00:33.842Z: Stopping worker pool...
Apr 30, 2020 7:02:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T19:02:38.281Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 30, 2020 7:02:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T19:02:38.329Z: Worker pool stopped.
Apr 30, 2020 7:02:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-30_11_59_00-16458478598023341465 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 6bdd8eff-edca-4cd7-9622-11072534e2cb and timestamp: 2020-04-30T19:02:46.161000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.499
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4 mins 6.492 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 13s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/oxqvhn3scdesk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #446
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/446/display/redirect>
Changes:
------------------------------------------
[...truncated 285.22 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 30, 2020 12:50:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 12:50:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 30, 2020 12:50:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 12:50:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 30, 2020 12:50:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 30, 2020 12:50:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-HUkIGH1qDc_Uab-ctvn8cg.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-R4iervM782T0gBjuxqWz9g.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-FhltxhzkpWpfR57EUfXw4A.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-6PAko61gJ4MEE_jE3uIFZQ.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-XvQkNZzcmvGlSN7PV9P2jA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-r33lSn__xZ2teXZl6y2QJw.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-D3KftjkRQ23EwSrL8-WPkA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-ZBZCs_sFZZp5maMPr9Xb6w.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-C2oynNFJWUYZfW9W8BQKuA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-4dzyCRAGZ9vKoAvLwclPXA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-lVZ6PckbkzfT0n0BLT4c0g.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-dKvI3Y5pVra4-i3HzwWV5g.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-tnthuYdta0-eNLGH_bBQ6w.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-VG2tylP0NNQEtYAyqdjnrw.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-5zAAzk_4ZtYal1DStYW3lA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-uy3azanoKTFsfDg3brTucg.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-zE28Vi8j24kgqzETCpeJrA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-dncLvHiX1cQbLTebGbZEhA.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-2LZM9wlAsmbaqim2J4DQ2A.jar
Apr 30, 2020 12:50:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-uvbldzl7gOIiZLPJTUhWrA.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-05dFqadTCpodQX43JrDvEA.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-86xNUAaB98BUrv-PPRAvmg.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-p0naZrA4OpY-AD1sqcrCUA.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-kOvqYdH0AGZD79s0KmeV9A.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-chvbc_iqzk3dr4FP4SG0FQ.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-qk4cjOeFZ4ru9Gs188iKcw.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-qk4cjOeFZ4ru9Gs188iKcw.jar
Apr 30, 2020 12:50:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-VTYIka_-2DmV5FrjqjXIhw.jar
Apr 30, 2020 12:50:57 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-qk4cjOeFZ4ru9Gs188iKcw.jar&uploadType=resumable&upload_id=AAANsUmwCLgs9VM4L4upX_yaHTT_7HjXnaj6CfHBEDWgXvVH3ZriEz4rfj9e6ormAJZplrXKBoUcp7jlM0Inr0E5j34.
Apr 30, 2020 12:50:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 30, 2020 12:51:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-qk4cjOeFZ4ru9Gs188iKcw.jar
Apr 30, 2020 12:51:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 30, 2020 12:51:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash yVdIWZJ9CRNQrDqIv0wZbw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yVdIWZJ9CRNQrDqIv0wZbw.pb
Apr 30, 2020 12:51:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 30, 2020 12:51:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-30_05_51_02-964206847272700150?project=apache-beam-testing
Apr 30, 2020 12:51:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-30_05_51_02-964206847272700150
Apr 30, 2020 12:51:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-30_05_51_02-964206847272700150
Apr 30, 2020 12:51:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T12:51:02.276Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 30, 2020 12:51:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:05.814Z: Checking permissions granted to controller Service Account.
Apr 30, 2020 12:51:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.141Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.749Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.790Z: Expanding GroupByKey operations into optimizable parts.
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.823Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.911Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.942Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:10.981Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:11.020Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:11.499Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:11.592Z: Starting 5 workers in us-central1-f...
Apr 30, 2020 12:51:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:37.571Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 12:51:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:37.608Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 30, 2020 12:51:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:43.032Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 12:51:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T12:51:43.116Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2020 12:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:56.132Z: Workers have started successfully.
Apr 30, 2020 12:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:51:56.169Z: Workers have started successfully.
Apr 30, 2020 12:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:52:37.683Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 12:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:52:37.879Z: Cleaning up.
Apr 30, 2020 12:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:52:37.971Z: Stopping worker pool...
Apr 30, 2020 12:53:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:53:57.820Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 30, 2020 12:53:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T12:53:57.874Z: Worker pool stopped.
Apr 30, 2020 12:54:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-30_05_51_02-964206847272700150 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a37e1c8f-3e56-44a4-a7c3-3e2a6750a978 and timestamp: 2020-04-30T12:54:03.046000000Z:
Metric: Value:
read_time 22.428
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 18.453 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 28s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ma6gppjzoblgw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #445
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/445/display/redirect?page=changes>
Changes:
[github] [BEAM-9561] Add a framework for running pandas doctests with beam
------------------------------------------
[...truncated 283.30 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 30, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-cjmJAW2TIEqxw9mSU3lU1w.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-T_VAIvMZpieh5U2vPZnr0A.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-86odgn8N_fT94oRQ7ShWBg.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-3PtroxVV2z0d3A_iNL44mg.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-yuPWQsuI_CmlQ6lS3BFhvw.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-scoqz4tkN-tjbwOZnoD29A.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-OcsZQBM4zDARpq6QgXlNow.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-n0InzPvprkUUsFMdyPua8w.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Qy_agO1yEBQNLARj3IWUsw.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-6XUcqDFlgTgPqrcQDGFVPA.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-dNMAc_9o03uOZ-oBWgRQ_A.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-_3ejUIlrcqr_kuVESoWYNA.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-ccehLfVwaR4DX7NcNfPTHA.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-u_bAG40NlW54UFY4m2Pfcg.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7_7QOd0zjeVSiY2DuX2byA.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Zo6LoG4upttZu_rp3NZF9Q.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-5g6IxoAGWZFy3UzQFbjpzw.jar
Apr 30, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Cp8uoeEngxzu-llMFa6ozA.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-3zlvspAC_soxIaTBRp1ffg.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-bfKIPwXP6jUU_iCNFIG3nQ.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-AhNv60mAtkAsRjmZ0fIjRw.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-ZmwpX4XHj8GDx6r65R6QOQ.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-nICV56vyegl_xbaZ2mpeZw.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-j0buSjS7CL9HJ0fRYe7ueA.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-12bRV-2PIuPRDPfNE6rwLw.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-P0zkGPWwVRILk2_yaVhZXA.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-rw7PIXbtVKOkL6Kb7bbOKA.jar
Apr 30, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-P0zkGPWwVRILk2_yaVhZXA.jar
Apr 30, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-P0zkGPWwVRILk2_yaVhZXA.jar&uploadType=resumable&upload_id=AAANsUk9syMLhsf6GGc3EC62b6ZouoYpjfTejfkXwIAtKhpUtNxQTc-Wz44z4iP7L9Zo6-tQhk6Y9sBRA9ipR9t-RQlP3MSfNA.
Apr 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-P0zkGPWwVRILk2_yaVhZXA.jar
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash E4N4M5kJZ9sv9mfE2Rh_Vg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-E4N4M5kJZ9sv9mfE2Rh_Vg.pb
Apr 30, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 30, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-29_23_45_37-9115957684767998186?project=apache-beam-testing
Apr 30, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-29_23_45_37-9115957684767998186
Apr 30, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-29_23_45_37-9115957684767998186
Apr 30, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T06:45:37.382Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 30, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:41.183Z: Checking permissions granted to controller Service Account.
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:45.038Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:45.773Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:45.835Z: Expanding GroupByKey operations into optimizable parts.
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:45.885Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.010Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.041Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.080Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.112Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.563Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:45:46.656Z: Starting 5 workers in us-central1-f...
Apr 30, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:46:12.339Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:46:12.369Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 30, 2020 6:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:46:17.750Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T06:46:20.279Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:46:33.375Z: Workers have started successfully.
Apr 30, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:46:33.416Z: Workers have started successfully.
Apr 30, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:47:08.752Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:47:08.969Z: Cleaning up.
Apr 30, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:47:09.062Z: Stopping worker pool...
Apr 30, 2020 6:48:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:48:48.868Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 30, 2020 6:48:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T06:48:48.928Z: Worker pool stopped.
Apr 30, 2020 6:48:54 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-29_23_45_37-9115957684767998186 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f6618fae-e921-4bd4-9ebc-ee1ae1c73db8 and timestamp: 2020-04-30T06:48:54.210000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.608
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 33.292 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 36s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/gu26rco3otfeu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #444
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/444/display/redirect?page=changes>
Changes:
[alxavier] [BEAM-9739]Retry SpannerIO write on Schema change
[lcwik] [BEAM-9846] Clean up references to unused native BQ source/sink
[github] Fix typo in go installation link. (#11445)
------------------------------------------
[...truncated 284.29 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 30, 2020 12:45:34 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 30, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 30, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-f3nAxDMAO4YK0QxoB7EcPQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-1jz2i356O4Zrksae3-Jbpw.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-kvP3S3ProfGHlArPv-Dn6w.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ak3vbafK9yCRfjcJOXoE3Q.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-odbsxMARDzxY71W81XmulQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-2DzKjacNTsoUxbhqimnOLg.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-GvMxp-vLUzJu85STJ8qNGg.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-YkZoi9UkgeVw7U3FOckHEw.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-HJjqGFQOieCRLVWj-92unQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-SGxB_cTlSzAI3NbIHR9ZbQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Ma7TksPohRQTd_NMV_TBag.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Y-73t9Msyqaw8TcQF1sK_w.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-oWd937pHanbz6ubC4A6c6g.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-gWi8ZF2E_YXPFeL9wMxK2Q.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-aDpj9M5pWkYv8k4EKecGSg.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-KEXClvhe09eRXnTNwySG-g.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-yv_c42BYfDTqz3gePUNZ4Q.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Dq2_MRELbKn3wRsM6tOByg.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-nMF-Wb8-xHhT8CX73Aw1gw.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-kJewnN3jN2K1Vm6_SFJ7BQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-GcGcN_7W_5et8q3UhaUs7Q.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-QZBVepMQnI7pdAfd4_RcsQ.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-vtKKP1Wk8DLlHQq0kZ7dOA.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Y5MFPi3up5460PaWjdIn0A.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-uChWX6Pd20vM5TIQjUJhPA.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uhOFmLXFzaE23eX0XaggpA.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-KNduqvlpKW30SsbL-ejD_g.jar
Apr 30, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uhOFmLXFzaE23eX0XaggpA.jar
Apr 30, 2020 12:45:37 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uhOFmLXFzaE23eX0XaggpA.jar&uploadType=resumable&upload_id=AAANsUlQOUDEy8wmemlE9CzehKZ4hr_c-xM0HOJxjjCZMMTR1YOAJVgIu16Jv_YvafSM7UR5BK1q5aRmJMNqsGUcId7cHb5CKw.
Apr 30, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 30, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-uhOFmLXFzaE23eX0XaggpA.jar
Apr 30, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash u6gul7CJC8C2Z8fg8GI6uQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-u6gul7CJC8C2Z8fg8GI6uQ.pb
Apr 30, 2020 12:45:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 30, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-29_17_45_47-8281267664350634708?project=apache-beam-testing
Apr 30, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-29_17_45_47-8281267664350634708
Apr 30, 2020 12:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-29_17_45_47-8281267664350634708
Apr 30, 2020 12:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T00:45:47.050Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 30, 2020 12:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:51.655Z: Checking permissions granted to controller Service Account.
Apr 30, 2020 12:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:55.813Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:56.730Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:56.797Z: Expanding GroupByKey operations into optimizable parts.
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:56.848Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:56.993Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:57.041Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:57.078Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:57.116Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:57.846Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:45:57.931Z: Starting 5 workers in us-central1-f...
Apr 30, 2020 12:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-30T00:46:19.837Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:46:26.652Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 12:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:46:26.686Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 30, 2020 12:46:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:46:32.078Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 30, 2020 12:46:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:46:50.843Z: Workers have started successfully.
Apr 30, 2020 12:46:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:46:50.883Z: Workers have started successfully.
Apr 30, 2020 12:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:47:26.082Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 30, 2020 12:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:47:26.284Z: Cleaning up.
Apr 30, 2020 12:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:47:26.367Z: Stopping worker pool...
Apr 30, 2020 12:48:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:48:53.432Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 30, 2020 12:48:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-30T00:48:53.479Z: Worker pool stopped.
Apr 30, 2020 12:48:59 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-29_17_45_47-8281267664350634708 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a03965ed-ba7f-4bf8-a75a-45a71df7cf37 and timestamp: 2020-04-30T00:48:59.529000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.935
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 34.288 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/uuokn4ecai4wg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #443
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/443/display/redirect?page=changes>
Changes:
[robertwb] [9577] Attempt to stage artifacts via new API on Java.
[robertwb] [BEAM-9802] Customization of automatically started services.
[dcavazos] [BEAM-9771] Fix notebook github URLs
------------------------------------------
[...truncated 284.74 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 29, 2020 6:49:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 29, 2020 6:49:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 29, 2020 6:49:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-D83f25gO_4W2G-k5guzoHQ.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-zmFsprdv0uP1_WmyjKFi7g.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-f14inTvsWNpJB0HgG6ErlA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT--9tRAlu8Aa6vavS26Cm9PQ.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-tUQoph-yIRHy_phA5H8AOw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-nArATFIUfgqCOglbCJtvvQ.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-esj6zWodhzBBRQxqcvmuOA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-l-9BeP5Xdix7YZ9rZFoYwg.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-8LgBmstovjQUp5dKtlY0pw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-jTI0vcatUwvddQ_j9FcRSw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-C0p75Ca_PqtLc2bn8uRnlA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-NoRDApccxXmd4_yQsEDnOA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-AfnVE3qFW88sgf0k1DNYag.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-lBaRjuVlL_aSbnI0adh3YQ.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Hr2jueuKYVMd-0JcbErjuA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-zuYIu9U_5ipy_S3Mlpi8-w.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-oAjfnKzQU4m7lZYs2ONQuw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-45I5YJX9kQ_AHwjSyIBlUA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Ku2oZnnlsVleJg91zv9Kug.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-k5I-0aAjAjCeN0CYcsKj2A.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-V77eK53CU7p_0tsVaz6sHA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-4ttUF6_YngEsx8YyLjPdsw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-5z0vn0Z3MZUIZGnA7snm3A.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-xTm8ImQS89a-afTSVjLKEw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-u0saSyGUYGAEEncNXRt0AA.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-49rMntfXsN6CG6HRUNkynw.jar
Apr 29, 2020 6:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-dHlhphlZQ1Y5SA9oGq7_ZQ.jar
Apr 29, 2020 6:49:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-u0saSyGUYGAEEncNXRt0AA.jar
Apr 29, 2020 6:49:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-u0saSyGUYGAEEncNXRt0AA.jar&uploadType=resumable&upload_id=AAANsUmH83Uxw_Cj2vORINpv0SHu9Is-pYh9qUrcUBlR5vOZo8LwTQ6Q3cnT08VOV-ISmnaLI9z7raIY26O2Fbs3AsLI6hzu4w.
Apr 29, 2020 6:49:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 29, 2020 6:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-u0saSyGUYGAEEncNXRt0AA.jar
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 29, 2020 6:49:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash oPZOdQoa_FkxcGSjCtskgw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-oPZOdQoa_FkxcGSjCtskgw.pb
Apr 29, 2020 6:49:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 29, 2020 6:49:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-29_11_49_34-4364244796599654196?project=apache-beam-testing
Apr 29, 2020 6:49:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-29_11_49_34-4364244796599654196
Apr 29, 2020 6:49:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-29_11_49_34-4364244796599654196
Apr 29, 2020 6:49:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T18:49:34.407Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 29, 2020 6:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:38.100Z: Checking permissions granted to controller Service Account.
Apr 29, 2020 6:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:41.133Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:41.982Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.022Z: Expanding GroupByKey operations into optimizable parts.
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.061Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.163Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.195Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.228Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 29, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.263Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 29, 2020 6:49:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.720Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 6:49:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:49:42.801Z: Starting 5 workers in us-central1-f...
Apr 29, 2020 6:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T18:50:00.223Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 29, 2020 6:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:10.967Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 6:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:11.013Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 29, 2020 6:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:16.417Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 6:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:28.904Z: Workers have started successfully.
Apr 29, 2020 6:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:28.943Z: Workers have started successfully.
Apr 29, 2020 6:51:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:58.590Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 6:51:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:59.032Z: Cleaning up.
Apr 29, 2020 6:51:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:50:59.133Z: Stopping worker pool...
Apr 29, 2020 6:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:52:39.589Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 29, 2020 6:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T18:52:39.641Z: Worker pool stopped.
Apr 29, 2020 6:52:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-29_11_49_34-4364244796599654196 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 04c52cb4-bba6-4a00-bfc6-1b01f72a136d and timestamp: 2020-04-29T18:52:45.548000000Z:
Metric: Value:
read_time 13.157
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 31.909 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/4c5kjicd2stbk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #442
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/442/display/redirect?page=changes>
Changes:
[github] [BEAM-9147] Make VideoIntelligence use PTransform on user-facing API
------------------------------------------
[...truncated 283.00 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 29, 2020 12:51:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-lFFauefS_IVxHTqVdmSVmA.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-OAy1Pdpk7ZSX5aB_dbrfZw.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Wwbs177ivE1ZZ2HRNqrMgg.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Mt_T4n4vMpaTqBNZfDe_3w.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-K_LzvOXsfymoA3PnaZiEtw.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Fv7r9RuM6OcMEJq3DI_HTg.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-kKJNY85G-Unzyx-LSiPtfg.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-jB39VzYElgDz7L8Oc0s0uA.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-_cTBV8odK6ZzpkBQdzWSlQ.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-zJg_rcPT0bTRIFMOnmBEgA.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-9bPAsgMN-wTUTyhtv12aSA.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT--n5UBM7HtCSPUoXBE4tNOQ.jar
Apr 29, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-O9yFFCuJMo_bq5XiFh6sZA.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-_vEMRYjgVLVocSiNOmCwEA.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Dsj-Gt7tACDLsDQ227nfKw.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-IPD_rID5pU66b8lS8vZSWw.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-_L5_bdcnhfADrlg1gz5lzw.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-NnvhY-UrcYyXESdT5xdycw.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-zba16BoEbBP0emVSS3yG-g.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-HaAw4cz7pyszkhwXMMzKCA.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-xqDJQ6yA_O6hmbw-L_Y2Og.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-MEKr05e87eGWkDkz146zmA.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-gNAuuzXusf4kfiNFyowxGQ.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-BHLXgdAR0yAIDFDf2GvwFg.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nfI_HU9jf3O6xlhoYNmxbQ.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Q5Jgk35bEALDay0ZW68WHg.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-m7eYPTexw6ZmAqJXg9n83w.jar
Apr 29, 2020 12:51:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nfI_HU9jf3O6xlhoYNmxbQ.jar
Apr 29, 2020 12:51:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nfI_HU9jf3O6xlhoYNmxbQ.jar&uploadType=resumable&upload_id=AAANsUmsRlTLjzyJaPdMtDXqu1BNglYF2mcD_TbhZtbNj_HcIOLaYKFDaVpGjPMUYG8IHsF9IOZanOjt1Fn6iRNQRuE.
Apr 29, 2020 12:51:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 29, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nfI_HU9jf3O6xlhoYNmxbQ.jar
Apr 29, 2020 12:51:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 29, 2020 12:51:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 29, 2020 12:51:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 29, 2020 12:51:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 29, 2020 12:51:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 29, 2020 12:51:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 29, 2020 12:51:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash Fps9jZzXEXQHESvmcSPmbw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Fps9jZzXEXQHESvmcSPmbw.pb
Apr 29, 2020 12:51:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 29, 2020 12:51:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-29_05_51_31-18380118856764517143?project=apache-beam-testing
Apr 29, 2020 12:51:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-29_05_51_31-18380118856764517143
Apr 29, 2020 12:51:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-29_05_51_31-18380118856764517143
Apr 29, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T12:51:31.730Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 29, 2020 12:51:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:34.913Z: Checking permissions granted to controller Service Account.
Apr 29, 2020 12:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.018Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.611Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.643Z: Expanding GroupByKey operations into optimizable parts.
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.685Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.773Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.806Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.841Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:39.878Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:40.238Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 12:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:51:40.333Z: Starting 5 workers in us-central1-f...
Apr 29, 2020 12:52:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T12:52:03.086Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 29, 2020 12:52:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:52:07.273Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:52:31.298Z: Workers have started successfully.
Apr 29, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:52:31.322Z: Workers have started successfully.
Apr 29, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:53:07.513Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:53:07.715Z: Cleaning up.
Apr 29, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:53:07.785Z: Stopping worker pool...
Apr 29, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:54:57.589Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 29, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T12:54:57.632Z: Worker pool stopped.
Apr 29, 2020 12:55:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-29_05_51_31-18380118856764517143 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8d60a68f-51e9-465b-a8ad-07499c612d7d and timestamp: 2020-04-29T12:55:03.570000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.297
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 48.418 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 48s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/vb5uzpdemc6bs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #441
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/441/display/redirect>
Changes:
------------------------------------------
[...truncated 283.49 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 29, 2020 6:50:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 29, 2020 6:50:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 29, 2020 6:50:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-oDCBpcGXEgWpHpLaBXugSA.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-pIZKDr_zwZrG_Lx-IPfYsQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-iRZEuPlUKdEp25pGky0PIQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-xtb2DgcgqTPuZXhLvpw3lQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-RjQPXX4w0QUZD6I76dFM0w.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-iXbnIHc2d-4YKWov-Sz2YQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-U9snCihTNVtwRLZBvBh4vQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Wh2RrlFKQ3QdZEsg6UiL3g.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-FESMGyesJPb--PjDAZskhQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--sL9TGCpaFyKeGQehvAj_Q.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-cb4hjBuEvh1SdwNAGii4xQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Qmapr3fFPMECgiYDMpV3_w.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-RtOZZyfvpH4g4QOnbeH6Dw.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-Nt3TcHm7K4pE1GFc5XbHWw.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-y_xLaXy2bSniun_l_XfFfg.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-u-p2sdBYCQHDSUpgRS5KAQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-N5MUEES30WYXVqmHtx5chQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-UuVV5zgLawe0UrBJvOr2cw.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-MWvxrKudGw-3wwpFRP1CjQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-b0hdYWNBifO2SHiFXFkcig.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-nqpeRzGPdjISWHf8AOPGow.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-YtNf8R69LguAvSg7QfFnIA.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-IJ_5JNpFPgJcALdR_wgFsQ.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-2XICKul8NRnWdj0aiP4jTw.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-rXsgHy9aKCTrAtPcb3RceA.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-2XICKul8NRnWdj0aiP4jTw.jar
Apr 29, 2020 6:50:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-2rwmAHJThSdVzRQcFhzcMw.jar
Apr 29, 2020 6:50:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-wK_y_JC4rf5wLFBweAilcg.jar
Apr 29, 2020 6:50:17 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-2XICKul8NRnWdj0aiP4jTw.jar&uploadType=resumable&upload_id=AAANsUmnXfrjKT7JKPDJfuC4vr0-vQejjFOWBCrk1x1iBM9MYjXklXFScfC97hItG_tMQBU2kSykEMLRF9nuLii90EQ.
Apr 29, 2020 6:50:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 29, 2020 6:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-2XICKul8NRnWdj0aiP4jTw.jar
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 29, 2020 6:50:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 29, 2020 6:50:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash Aymns8zPuNwBHygD7hZ-zw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Aymns8zPuNwBHygD7hZ-zw.pb
Apr 29, 2020 6:50:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 29, 2020 6:50:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-28_23_50_26-3242202383340423429?project=apache-beam-testing
Apr 29, 2020 6:50:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-28_23_50_26-3242202383340423429
Apr 29, 2020 6:50:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-28_23_50_26-3242202383340423429
Apr 29, 2020 6:50:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T06:50:26.691Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 29, 2020 6:50:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:30.095Z: Checking permissions granted to controller Service Account.
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:34.137Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:34.827Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:34.871Z: Expanding GroupByKey operations into optimizable parts.
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:34.899Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.000Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.028Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 29, 2020 6:50:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.067Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 29, 2020 6:50:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.100Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 29, 2020 6:50:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.601Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 6:50:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:50:35.689Z: Starting 5 workers in us-central1-f...
Apr 29, 2020 6:50:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T06:50:40.945Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 29, 2020 6:51:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:51:07.448Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 6:51:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:51:30.708Z: Workers have started successfully.
Apr 29, 2020 6:51:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:51:30.742Z: Workers have started successfully.
Apr 29, 2020 6:52:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:52:02.928Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 6:52:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:52:03.129Z: Cleaning up.
Apr 29, 2020 6:52:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:52:03.204Z: Stopping worker pool...
Apr 29, 2020 6:53:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:53:53.700Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 29, 2020 6:53:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T06:53:53.741Z: Worker pool stopped.
Apr 29, 2020 6:53:59 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-28_23_50_26-3242202383340423429 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): bb9959fe-ced4-4179-8755-4be3e9d691cd and timestamp: 2020-04-29T06:53:59.325000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.653
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 53.15 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 5s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/x22qnyiy4ducm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #440
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/440/display/redirect?page=changes>
Changes:
[robertwb] Conditionally cleanup tempdir.
[mxm] [BEAM-6661] Properly close channel for external environment
[mxm] [BEAM-6661] Avoid FileNotFoundException when no files have been staged
[mxm] [BEAM-6661] Avoid printing empty lines in subprocess job server
[robertwb] Use logging to reduce verbosity of license checking when all goes well.
[github] All Python DirectRunners call setup/teardown on DoFns (#11547)
[github] [BEAM-9815] Correct go integration test if clauses. (#11524)
[relax] Support schemas in state API
[github] [BEAM-7923] Support docker screendiff integration tests (#11507)
------------------------------------------
[...truncated 286.63 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 29, 2020 1:23:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-mJbwbDyxM4OPa4F8o2JCfw.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tMWBIiNQNMxbEN_N5UqwFQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-mlCKpUcu_kubeQd0OcPYvg.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-NvkvytO3dN3Kw_-eoJCpCQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-oFDmFFG-yi3yPQpH62io0Q.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-cclO_ZecmOIRcUkCWL8yIg.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ky_FA1whbBVB5XEKkd6FHA.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-AMIzJ-IIWBpxYAFDUmAHew.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-jYujqH6u_X1cA0Jwkb5d0Q.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-2ktaToTnV89Snxv3pS6wVg.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-lHaXE0cZjUTQcu0cWJuNYQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-GW41MgHRkG4VFruIhFJ0xQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-lMJcfLbOkJFkEHIf8Gcepg.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Ocj8KU_n4vTd4bgjqaCGIQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-DPMB-74S1THPlGJSMmZgzQ.jar
Apr 29, 2020 1:23:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-jHmRzx16LnbywNmmKuX-PQ.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-DsMQuZj4S77ovnbfKyfG2w.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-kejZgqs0e-PZEzk9MaYSFw.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-CZaAiXTUz4kSVetbptawMA.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-aGnlqu-By5iEo9lh_04YYg.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-hIOj6M8_mV-l3yj-pPi2sw.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-4Vu30jGWNpH1cuvrWlyM3A.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-PvRhvBe7nl0vFws2-rY88g.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nV-F4KxfJ6ll8fnS7TgBMw.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-lHAd4x_9brWfzwP4Dft3rQ.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nV-F4KxfJ6ll8fnS7TgBMw.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT--wbNfcWbf0rrKhfxuGD4eA.jar
Apr 29, 2020 1:23:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-JVFCR-5Z4EE0Bf0nWYKLIw.jar
Apr 29, 2020 1:23:33 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nV-F4KxfJ6ll8fnS7TgBMw.jar&uploadType=resumable&upload_id=AAANsUn9yQkmXiQZqfS5_hsJU7cWplORT6q-dJKjy8lv2cpfX5c0xLO1FQ9DkvoE-pPfBd72Az1evh749Y15RUfKvnasojFJOA.
Apr 29, 2020 1:23:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 29, 2020 1:23:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-nV-F4KxfJ6ll8fnS7TgBMw.jar
Apr 29, 2020 1:23:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash g_5bZlBmOdLrpoPMQWZ6Lw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-g_5bZlBmOdLrpoPMQWZ6Lw.pb
Apr 29, 2020 1:23:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 29, 2020 1:23:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-28_18_23_38-120377818692259132?project=apache-beam-testing
Apr 29, 2020 1:23:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-28_18_23_38-120377818692259132
Apr 29, 2020 1:23:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-28_18_23_38-120377818692259132
Apr 29, 2020 1:23:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T01:23:38.092Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 29, 2020 1:23:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:41.705Z: Checking permissions granted to controller Service Account.
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.175Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.787Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.818Z: Expanding GroupByKey operations into optimizable parts.
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.854Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.925Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.959Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:45.996Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:46.030Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:46.436Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 1:23:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:23:46.517Z: Starting 5 workers in us-central1-f...
Apr 29, 2020 1:24:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:24:16.387Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 1:24:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:24:16.426Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 29, 2020 1:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-29T01:24:20.631Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 29, 2020 1:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:24:22.288Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 29, 2020 1:24:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:24:37.374Z: Workers have started successfully.
Apr 29, 2020 1:24:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:24:37.409Z: Workers have started successfully.
Apr 29, 2020 1:25:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:25:14.064Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 29, 2020 1:25:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:25:14.277Z: Cleaning up.
Apr 29, 2020 1:25:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:25:14.362Z: Stopping worker pool...
Apr 29, 2020 1:27:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:27:23.091Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 29, 2020 1:27:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-29T01:27:23.139Z: Worker pool stopped.
Apr 29, 2020 1:27:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-28_18_23_38-120377818692259132 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 34fc1753-561a-4abf-b49c-37a7c6d01425 and timestamp: 2020-04-29T01:27:30.156000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.857
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 8.424 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 37s
101 actionable tasks: 77 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/rxm355hfsuwdm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #439
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/439/display/redirect?page=changes>
Changes:
[github] Display correct package in help message
[apilloud] [BEAM-9658] Plumb through WITH OFFSET
[kcweaver] [BEAM-9758] Use quotes in echo statements to prevent * expansion.
[github] [BEAM-9443] update document about direct_num_workers=0 for FnApi
[github] Speedup on FnApiRunner start up time (#11543)
------------------------------------------
[...truncated 286.23 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 28, 2020 8:13:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 28, 2020 8:13:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 28, 2020 8:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-2cKrW8_B62oVMU91n6yB9g.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-F1PXI5wrxpaStiWSNxd0aA.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-HMvrzYe84d9H3gVdPRdmlw.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-PUKLXiqQw-W57DAX7rg5Ng.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Hhctyu7zcJbSfh0HRbHENg.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-xvVlvgZnE5sFzu0RFY5uKg.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-MGehT-aks3sP2gpRrCBP6w.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-xcC6uuS7pJygk9hSK4fH3w.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Q8WCwBKGHoRu8UWYh58LBA.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-3mYGWpoHKb0-ZOj4lIPPAw.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-_dkM67ZuT-OKZjSUrCp2FQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-0AC8fSHHNAdDJlAelOUmfQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-eXXQ-X9fd3lUym0pvh4Sdw.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-61TXGR6j1s9zq7UvpZ2UiQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-CEAmRIHBQ45wKygGYIO9PQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-k-lEah8o8lqc4cnmxqQ4Ig.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-olX8XZ1kpEMMrl2ydfxVpw.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-VbDvrDtdICzaUQNb7mZFSQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-61TXGR6j1s9zq7UvpZ2UiQ.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-x_7eZ488VEnMxcdEO_JFkg.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-DzmNNBSnsodM5E0Stqeo1g.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-gnX1pB2Q9r5LXVjlXcni7g.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-4ihu-ps4hNcoULAmJ4YiXA.jar
Apr 28, 2020 8:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-owXkt9gOSjZqPepgkeJG-w.jar
Apr 28, 2020 8:13:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-gU7TY_-dLuUkOukFqZppDw.jar
Apr 28, 2020 8:13:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ckRHT0wmoFKlPjdg7NsJDQ.jar
Apr 28, 2020 8:13:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-7JDC86MYxL9fMLSsOkJN_g.jar
Apr 28, 2020 8:13:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-XZy83ad0pd7K8zRkoFdtJQ.jar
Apr 28, 2020 8:13:23 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-61TXGR6j1s9zq7UvpZ2UiQ.jar&uploadType=resumable&upload_id=AAANsUlwH34t2WrtdhXqaFGqdyTjQCZkej7QPH9lhbelRDk9XCytnzYxsffSYP3hh-0G-FuHwSXvPrq6h7Ul-Wy2u-0AliEurw.
Apr 28, 2020 8:13:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 28, 2020 8:13:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-61TXGR6j1s9zq7UvpZ2UiQ.jar
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 28, 2020 8:13:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash aKx8irugoXTU6c28kao7Ow> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aKx8irugoXTU6c28kao7Ow.pb
Apr 28, 2020 8:13:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 28, 2020 8:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-28_13_13_31-125045480358135695?project=apache-beam-testing
Apr 28, 2020 8:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-28_13_13_31-125045480358135695
Apr 28, 2020 8:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-28_13_13_31-125045480358135695
Apr 28, 2020 8:13:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T20:13:31.434Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 28, 2020 8:13:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:35.439Z: Checking permissions granted to controller Service Account.
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:42.747Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.354Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.411Z: Expanding GroupByKey operations into optimizable parts.
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.447Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.550Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.592Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 28, 2020 8:13:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.638Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 28, 2020 8:13:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:43.682Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 28, 2020 8:13:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:44.154Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 8:13:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:13:44.236Z: Starting 5 workers in us-central1-a...
Apr 28, 2020 8:14:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T20:14:01.143Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 28, 2020 8:14:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:14:10.842Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 8:14:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:14:10.884Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 28, 2020 8:14:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:14:16.265Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 8:14:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:14:28.658Z: Workers have started successfully.
Apr 28, 2020 8:14:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:14:28.709Z: Workers have started successfully.
Apr 28, 2020 8:15:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:15:01.649Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 8:15:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:15:01.888Z: Cleaning up.
Apr 28, 2020 8:15:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:15:02.018Z: Stopping worker pool...
Apr 28, 2020 8:16:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:16:58.706Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 28, 2020 8:16:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T20:16:58.770Z: Worker pool stopped.
Apr 28, 2020 8:17:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-28_13_13_31-125045480358135695 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8a7b3a68-18a5-41eb-99e5-ef53d67c28ad and timestamp: 2020-04-28T20:17:05.514000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.294
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 57.1 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 44s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/xg7krig6z3plu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #438
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/438/display/redirect?page=changes>
Changes:
[kamil.wasilewski] [BEAM-8790] Kubernetes deployment for InfluxDB
[kamil.wasilewski] [BEAM-8790] Local deployment of InfluxDB with docker-compose.yml
[david.moravek] [BEAM-9824] Do not ignore chained Reshuffles on flink batch runner.
------------------------------------------
[...truncated 282.17 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 28, 2020 1:06:33 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 28, 2020 1:06:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 28, 2020 1:06:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-l1pVpeV64bdesl6eLjdcgQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-eUkcVQ_KVpt0yVK2o0Zfaw.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-oJzQgrocd3ZAu1TLOpmMnw.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aqmd5xU9UMwpGi491nTN2g.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-fvOfepgdAAu4FurX17llfQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ScllpE8uEZWNxZ5f0VYthA.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT--FY9RmGND4991YhzVj0Ftg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-JxkcFQhpqFNNRFuRV5sxHw.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-qE2z0mbVKt1lpP8QuCgK1g.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-cX7oFiCIyJCeNkZsiniMCg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-CiCmVY2UrlWpNSOFDbECHg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-lk08cS6Nlqgogr1yx4kxuQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-Gaf8KOYB9dE1p95PrSiCNw.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-8DooppBpp-8Um1M0Ok3sTA.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-gorheeR4QhNgYYaSSR_icQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-r9ItcpCMCmJ9xfPv-wG34w.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-iJ8v6DiK39Vl0gxdFlNlkA.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-TZNfyt3Y9kPw40w6dnL6NA.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-X6FLnoPvbk072G2RgY7uTQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-cUBKGCFxQJUdeCjH7j4oYQ.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-TMy_bhZYuEoSaOnP9OxKLw.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-_Qki8JsYe77z7_3ZeaGa6Q.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-zzzONcjFwVoXnVHxpKuGJA.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-pKrqsmHKIVxh7L42saOg7w.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-vZg_5N5oaupqtNTcfXsZyg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-JQmt91XctnW-_im4R6BOjg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-1_FB4KmbaE7Frc-xPjR8Qg.jar
Apr 28, 2020 1:06:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-JQmt91XctnW-_im4R6BOjg.jar
Apr 28, 2020 1:06:36 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-JQmt91XctnW-_im4R6BOjg.jar&uploadType=resumable&upload_id=AAANsUnWCwGWNXwEhD1_vXMcp11geKVmpqdn_Zxek3mkMDVTedFAy1RWqOk_hV9i7MxnElHY-IRFDk1MMuNsSA6DX-g.
Apr 28, 2020 1:06:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 28, 2020 1:06:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-JQmt91XctnW-_im4R6BOjg.jar
Apr 28, 2020 1:06:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash cZoqlSQYQUTvY1WyzO0-6A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cZoqlSQYQUTvY1WyzO0-6A.pb
Apr 28, 2020 1:06:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 28, 2020 1:06:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-28_06_06_44-16579986606783876265?project=apache-beam-testing
Apr 28, 2020 1:06:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-28_06_06_44-16579986606783876265
Apr 28, 2020 1:06:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-28_06_06_44-16579986606783876265
Apr 28, 2020 1:06:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T13:06:44.101Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 28, 2020 1:06:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:47.477Z: Checking permissions granted to controller Service Account.
Apr 28, 2020 1:06:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:51.491Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.231Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.292Z: Expanding GroupByKey operations into optimizable parts.
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.345Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.449Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.494Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.543Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:52.583Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:53.155Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 1:06:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:06:53.234Z: Starting 5 workers in us-central1-f...
Apr 28, 2020 1:07:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T13:07:17.814Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 28, 2020 1:07:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:07:29.585Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 1:07:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:07:50.181Z: Workers have started successfully.
Apr 28, 2020 1:07:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:07:50.221Z: Workers have started successfully.
Apr 28, 2020 1:08:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:08:21.065Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 1:08:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:08:21.317Z: Cleaning up.
Apr 28, 2020 1:08:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:08:21.427Z: Stopping worker pool...
Apr 28, 2020 1:10:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:10:08.917Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 28, 2020 1:10:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T13:10:08.970Z: Worker pool stopped.
Apr 28, 2020 1:10:14 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-28_06_06_44-16579986606783876265 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 4eefded3-7428-4b70-89d4-c6eac5f7399d and timestamp: 2020-04-28T13:10:14.352000000Z:
Metric: Value:
fields_read 4375276.0
read_time 12.978
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 50.266 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 53s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ykgn4gtddus4u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #437
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/437/display/redirect?page=changes>
Changes:
[github] Merge pull request #11546 from [BEAM-9832] Ensuring tags are not added
------------------------------------------
[...truncated 282.75 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 28, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-DVjuIJQoQTfxYJg-z0P9Yg.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Bh0XHKQmWItg20mVz8Fh7Q.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-ZMoMJNZBjgPVA680k0L4pw.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-QkUqWOhjrQhISVJY4mqy-w.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-XK6BRQfx5Yo-Hpw3HSJ5Rg.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-i8-lb4ReglXVg5RayXQVgA.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bfQnzYnG402e3Yrb2x0j8w.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests--f2FH-8SqxEHmNXXwDS_Aw.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-SEEuDIQtyOHQ8cjSRVvnhA.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Bw_g2I7j7kOl5kIQ_mkuRQ.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-VeMixsfs7dEXwjzB5EoLLQ.jar
Apr 28, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-lG2RHaS0TgxlLDtOwTfF1w.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-E8IPkdT0dZxEi5MjdpPUyw.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-3HNY5Lq-nJ43Lj9pWYa_8A.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-na5gFkKWL4k17TZ_1ZhLgw.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-k-e6Is73tYDj0vtjyk8R4Q.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-jGwib_yD2TdW0NM0M06V-g.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-fr0eNSbbo3hlvn_hkhEbxA.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-e65BZC_exjKwnb3wda9xVQ.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-xsIn4SYjROBVhYm_qG6Wcg.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-hnaFp0-RAPxwmkUhqc6CVA.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-tF5a5MIj_4IHO9gCVkeglA.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-L1Xw06EZ8xUoa0ST-DygpQ.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-lGCvLmRVTSnxhPj-NXyJJw.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mIZv2sF1uKOo6eM4Oh7pgw.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-avbvTUL2RxR2PVjycOYp2g.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mIZv2sF1uKOo6eM4Oh7pgw.jar
Apr 28, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-P2LaNV71YcATL2EeMkKovA.jar
Apr 28, 2020 6:45:29 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mIZv2sF1uKOo6eM4Oh7pgw.jar&uploadType=resumable&upload_id=AAANsUlR9LpDAtVkHhCcXx6byaP1t4dyKGwsBkaiKf8t3Rbs6SpJXyo773SjWyfC8zQMNErMHMcSyFYwboM-NSWxT3o.
Apr 28, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 28, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mIZv2sF1uKOo6eM4Oh7pgw.jar
Apr 28, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 2o3s5UJq9BwnVe06q_lL1A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2o3s5UJq9BwnVe06q_lL1A.pb
Apr 28, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 28, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-27_23_45_37-18371980610049167583?project=apache-beam-testing
Apr 28, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-27_23_45_37-18371980610049167583
Apr 28, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-27_23_45_37-18371980610049167583
Apr 28, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T06:45:37.901Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 28, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:41.495Z: Checking permissions granted to controller Service Account.
Apr 28, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.043Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.621Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.671Z: Expanding GroupByKey operations into optimizable parts.
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.715Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.824Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.857Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.901Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:46.933Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:47.315Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:45:47.399Z: Starting 5 workers in us-central1-f...
Apr 28, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:46:11.826Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:46:11.866Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 28, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T06:46:11.990Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 28, 2020 6:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:46:17.256Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:46:32.826Z: Workers have started successfully.
Apr 28, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:46:32.858Z: Workers have started successfully.
Apr 28, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:47:03.825Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:47:04.059Z: Cleaning up.
Apr 28, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:47:04.145Z: Stopping worker pool...
Apr 28, 2020 6:48:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:48:44.327Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 28, 2020 6:48:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T06:48:44.376Z: Worker pool stopped.
Apr 28, 2020 6:48:51 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-27_23_45_37-18371980610049167583 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 921d21b2-19c9-4543-a903-c8c0a13811ef and timestamp: 2020-04-28T06:48:51.997000000Z:
Metric: Value:
fields_read 4375276.0
read_time 12.745
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 34.133 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 34s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/mqsbw2ix6klpe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #436
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/436/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9807] [BEAM-9808] Add missing region to Dataflow examples.
[kcweaver] Missed a few spots.
[kcweaver] Only add --region for Dataflow runner.
[github] Updates Dataflow Python containers to 20200423 (#11535)
[bhulette] Fix JavaSDK typo
[bhulette] Add Python SqlTransform to CHANGES.md
[github] [BEAM-9779] Patch HL7v2IOWriteIT Flakiness (#11450)
[github] virtualenv (#11502)
[kcweaver] [BEAM-9833] Add .asf.yaml file.
[kcweaver] Send issues to github@
[kcweaver] Add license header.
------------------------------------------
[...truncated 284.96 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 28, 2020 12:46:59 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-kSAqKiqAMkTyqwehUrKKrQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aJlqyuMiPT3BLHevyjwMWQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-wI_zywsylcXvzevr4Dw2zQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-6x04M3C6hA_Gc2H_g-6SVg.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-w729yQuvnF_tWge5Ekf2xQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-evOJ_wW4m-61LXn8JS33rg.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-4bVb7fIfpoeEV3gpW6ZNyg.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-EEBqm1Vw_SdPrd7BYIyIGA.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-vcjpR1GLYpz1HapqJyzp4A.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tmyenRiq5pplcgRU-2shcQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-eLBk_wNDzEhngUptRV_dCg.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-XpgjjZmjWt0u6p2U-qLn4g.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-P4d1Gw8NBw18IJUJ9TPtUg.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-K74ukqIitCodit7fSNz7Uw.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-8x65KAjM3WquCHYPZf2TAA.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-PvJlR7nwbFM4JZ1lb3-apQ.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-yo5RmG-S7VpbQeyC2GnbMA.jar
Apr 28, 2020 12:47:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-vdCqrEvc_xcf9_IcQcuUBQ.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-nbtPuer1sCJAbsWPpMhecg.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-VYrUIk2J4LsWPCv2fK0jwQ.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-WqEK7J8QAJI69oU5X83cwA.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-EshNk6KDFIc11dlpml0joA.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-SdJOHQjAPLwauAR8WFKwjQ.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-uDzcFG-kwY3FzsfX147ggw.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-U2-NkEpvZjtpDSVAMgqq2A.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Qriuauh9Od84E36MbT_zRQ.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Q9ROun3F7pd3ZTDfKH5Svw.jar
Apr 28, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Q9ROun3F7pd3ZTDfKH5Svw.jar
Apr 28, 2020 12:47:02 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Q9ROun3F7pd3ZTDfKH5Svw.jar&uploadType=resumable&upload_id=AAANsUkiAwBz0cthI0rgShCz3znO2lIlghemcJL5YcvMIjfD8sxmKZ5kYAJ4Jz2mMdF-A7lJ-WjM4thXC7lX3MAe664.
Apr 28, 2020 12:47:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 28, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Q9ROun3F7pd3ZTDfKH5Svw.jar
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 28, 2020 12:47:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash S4mwfAMP0tQllDE2-G9lRw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-S4mwfAMP0tQllDE2-G9lRw.pb
Apr 28, 2020 12:47:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 28, 2020 12:47:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-27_17_47_11-9288545004707101039?project=apache-beam-testing
Apr 28, 2020 12:47:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-27_17_47_11-9288545004707101039
Apr 28, 2020 12:47:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-27_17_47_11-9288545004707101039
Apr 28, 2020 12:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T00:47:11.242Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 28, 2020 12:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:15.133Z: Checking permissions granted to controller Service Account.
Apr 28, 2020 12:47:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:18.361Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.297Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.339Z: Expanding GroupByKey operations into optimizable parts.
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.367Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.433Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.467Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.500Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.533Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.900Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:19.964Z: Starting 5 workers in us-central1-f...
Apr 28, 2020 12:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-28T00:47:30.830Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 28, 2020 12:47:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:47.838Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 12:47:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:47.877Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 28, 2020 12:47:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:53.320Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 12:47:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:53.353Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 28, 2020 12:47:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:47:58.881Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 28, 2020 12:48:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:48:07.894Z: Workers have started successfully.
Apr 28, 2020 12:48:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:48:07.936Z: Workers have started successfully.
Apr 28, 2020 12:48:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:48:45.119Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 28, 2020 12:48:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:48:45.324Z: Cleaning up.
Apr 28, 2020 12:48:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:48:45.418Z: Stopping worker pool...
Apr 28, 2020 12:50:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:50:37.369Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 28, 2020 12:50:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-28T00:50:37.422Z: Worker pool stopped.
Apr 28, 2020 12:50:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-27_17_47_11-9288545004707101039 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0a43f99d-7003-4de0-ba66-40f077cd793b and timestamp: 2020-04-28T00:50:43.233000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.476
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.045 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 52.668 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 9s
101 actionable tasks: 70 executed, 31 from cache
Publishing build scan...
https://gradle.com/s/wbcgnuj5yo7he
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #435
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/435/display/redirect?page=changes>
Changes:
[coheigea] BEAM-9826 - Update Tika to 1.24.1
[mxm] [BEAM-9827] Ensure minimum watermark hold is computed across all keys
[github] Merge pull request #11154: [BEAM-1819] Key should be available in
------------------------------------------
[...truncated 284.14 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 27, 2020 6:54:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 27, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 27, 2020 6:54:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-gLHeJ0JaiwtfY_XkDRvEng.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-ezwbtCYx7s3mXGDCqysNuA.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-G2w-smihtNRCEGeQ9B9dsg.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-824SVuebVCMfMUpGtWQ-Lw.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-FvOS4OWJ2qBbUYQJLTyrng.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-KiQUTaFTxQzME23NWEaR4w.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-7z6SMB08GGNNyFtcpxinMA.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-y2ZDaS5ZHk-A9KZRwQ2b6w.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-w05ZNWaXrExpKRsyptY0AA.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-v3ePEI7kaI-n9HTGCoLf-g.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-6HylE5yVFHFRGLN3ZkTh6A.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-CCSFDEGeKEXB63L4EjGq2A.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-b0Aown1NTpT9dbtZjBKICA.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-P-aw7FnuXvPvkncufGmTww.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-L4GXFCaNGpHojlzmZjHqeA.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-xj-KxQYFhkjXFYyXc9bQVw.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ZGMWa4umRU0b6Ci3AKZV3w.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-4KF3zw-4HhaW2yPhy6pr8A.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-kR8er5r_KX0ZiyMie39QIQ.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-5xwLsZE_6SdbbfUdOHzc1w.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vx9UwvHgjsknHMvowFiW3w.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-ZlpnV6aBGomZ7ZHC33SgNw.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-6yso1PviixJiOdhZPQhQxQ.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-pv0FWeR1z5EfAOJr7I5FDQ.jar
Apr 27, 2020 6:54:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-_akVoeNq-C9l6hSIWcpEiQ.jar
Apr 27, 2020 6:54:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vx9UwvHgjsknHMvowFiW3w.jar
Apr 27, 2020 6:54:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Mz4cqBIQbGeq-F83wQIq_g.jar
Apr 27, 2020 6:54:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-f4XNq204FCm0AxZQZiQTUw.jar
Apr 27, 2020 6:54:57 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vx9UwvHgjsknHMvowFiW3w.jar&uploadType=resumable&upload_id=AAANsUlTJNc2je22avAJQjXbjMN6Ts_yj8BrAN6lU__g6gqFP29nHGZpwNQanvdB9VSO54khKU17-zSvgLksFk_ZXW0.
Apr 27, 2020 6:54:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 27, 2020 6:55:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vx9UwvHgjsknHMvowFiW3w.jar
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 27, 2020 6:55:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 7wm-af6DsxM8wGf2g4X2gw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7wm-af6DsxM8wGf2g4X2gw.pb
Apr 27, 2020 6:55:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 6:55:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-27_11_55_05-2493382971476328337?project=apache-beam-testing
Apr 27, 2020 6:55:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-27_11_55_05-2493382971476328337
Apr 27, 2020 6:55:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-27_11_55_05-2493382971476328337
Apr 27, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T18:55:05.218Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 27, 2020 6:55:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:09.227Z: Checking permissions granted to controller Service Account.
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:13.177Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:13.798Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:13.842Z: Expanding GroupByKey operations into optimizable parts.
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:13.873Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:13.969Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:14.004Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:14.039Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 27, 2020 6:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:14.074Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 27, 2020 6:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:14.524Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 6:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:14.611Z: Starting 5 workers in us-central1-f...
Apr 27, 2020 6:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T18:55:40.115Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2020 6:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:43.514Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 6:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:43.550Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 27, 2020 6:55:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:55:54.559Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 6:56:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:56:03.187Z: Workers have started successfully.
Apr 27, 2020 6:56:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:56:03.229Z: Workers have started successfully.
Apr 27, 2020 6:56:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:56:42.429Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 6:56:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:56:42.620Z: Cleaning up.
Apr 27, 2020 6:56:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:56:42.705Z: Stopping worker pool...
Apr 27, 2020 6:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:58:52.167Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 27, 2020 6:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T18:58:52.207Z: Worker pool stopped.
Apr 27, 2020 6:58:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-27_11_55_05-2493382971476328337 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2a1ca21f-2fbf-426b-b606-281f9402604e and timestamp: 2020-04-27T18:58:58.340000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.421
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.196 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4 mins 12.909 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 26s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/fwngik24fqzoa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #434
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/434/display/redirect>
Changes:
------------------------------------------
[...truncated 284.19 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 27, 2020 12:51:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 27, 2020 12:51:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 27, 2020 12:51:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-X3R0IhWawZYQKrdiqjfAaQ.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ISmeiAWXVI5qvRburWmNWw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-iwzHw2s5NtaE44tWFyiuNQ.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-esWQ5HykGnmEp7qdNmOfsQ.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Y8-9-M1a0ov1ViBpesfM0A.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-opuL_SUulLTQ4hk1NENHMg.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-SlJFJgDri317pauKcjgsaA.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-SXBZaqkfLqtcZuPKuhsf0Q.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Tbr8wHmiBA_8TpS1eEcq6A.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-AXRIhVlETmkS4X9fzQHNnw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-TRWJZpFcWT26QUisUFIPdQ.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-taD9zDedSxMDj6pALHv-gQ.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-HTYz8u7GgYBLfaFEFNzoEA.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-ry74OLe7VPg6cwWADRqeXw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-jKWIs0uudDKoT2dDBRp9EA.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-1zCgrECqpFgTgozEfYjWHg.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-o6FlwXfntcbdkJKWhdiV1w.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-wzoqDkEbnEvLe_7Xb89L-w.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-__PIGKF3j8w9ZO6teh-oCA.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Fb8DBBwQ0-QM9Rs95alGFw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-0h9hW-bdImVFIGaksDOn9Q.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-JF_zAf66YCHP_seb-p2Maw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-yPYPMm8qdU0xcNpIqqUGiA.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-YVxqIlTxFPGPhiYMuqBbnw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-uHhxKUUIdBv9FycliEGWLw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FHL7GJcogVpeyw2a3Po3cw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FHL7GJcogVpeyw2a3Po3cw.jar
Apr 27, 2020 12:51:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-dytMn34JmZUvfzWE8hpKEw.jar
Apr 27, 2020 12:51:13 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FHL7GJcogVpeyw2a3Po3cw.jar&uploadType=resumable&upload_id=AAANsUmUQ2uqVaW70p8DFIMZIaDfeJXesVQXRu3lYzCN7jhKzp7EEnhPCo7ftwbP3rzPLfx3mKykgImC8GTQQC4VrOk.
Apr 27, 2020 12:51:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 27, 2020 12:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-FHL7GJcogVpeyw2a3Po3cw.jar
Apr 27, 2020 12:51:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash vHbWSuwY0PJX3dK8x7X5Gg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vHbWSuwY0PJX3dK8x7X5Gg.pb
Apr 27, 2020 12:51:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-27_05_51_23-16731973259693052710?project=apache-beam-testing
Apr 27, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-27_05_51_23-16731973259693052710
Apr 27, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-27_05_51_23-16731973259693052710
Apr 27, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T12:51:23.052Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 27, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:26.183Z: Checking permissions granted to controller Service Account.
Apr 27, 2020 12:51:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:29.687Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 27, 2020 12:51:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.400Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.444Z: Expanding GroupByKey operations into optimizable parts.
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.482Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.557Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.593Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.617Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:30.653Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:31.022Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:31.092Z: Starting 5 workers in us-central1-f...
Apr 27, 2020 12:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:58.861Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 12:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:51:58.895Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 27, 2020 12:52:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T12:52:01.805Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2020 12:52:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:04.340Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 12:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:17.301Z: Workers have started successfully.
Apr 27, 2020 12:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:17.327Z: Workers have started successfully.
Apr 27, 2020 12:52:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:51.726Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 12:52:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:51.925Z: Cleaning up.
Apr 27, 2020 12:52:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:52:52.015Z: Stopping worker pool...
Apr 27, 2020 12:54:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:54:15.908Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 27, 2020 12:54:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T12:54:15.954Z: Worker pool stopped.
Apr 27, 2020 12:54:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-27_05_51_23-16731973259693052710 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d18e590b-1eb1-43e8-8e34-211f44d71f17 and timestamp: 2020-04-27T12:54:22.916000000Z:
Metric: Value:
read_time 16.941
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 21.126 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/vk6jpbs5q2g5c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #433
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/433/display/redirect>
Changes:
------------------------------------------
[...truncated 283.30 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 27, 2020 6:54:59 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 6:54:59 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 27, 2020 6:54:59 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 6:54:59 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 27, 2020 6:55:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 27, 2020 6:55:00 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 27, 2020 6:55:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 27, 2020 6:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-y_Rv_NaZCynyIxqdQ_ztdQ.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-dRyn37zRChU7-2Bpg8WtcA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Qzxi-osNeRGyF8D1nrpOnw.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-zkhkqx5G8zg3zoDC9R9mhg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-cVtg0evNaTLdGG279xKIAA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-bW7QPX1sN7Laa7NlsePX1w.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-ubjFvVRKnpiXPETY2Si7GA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-3ZiHsJmicr93LcnFJ2gxCg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-OYoS2Su7X6QK4Wr_kvaqDg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-0CP666ZDTNxBuV3VsNecJA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-NVkseoRAuCbRMvAyzXbXYg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-5-Rrdu4cobXyqP2mD7VgQQ.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-3rSGdZrCQzEjlyD2GARPtA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-K-kO2Kh_DlfrphDlW49vtQ.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-MuTB2mOp4EBoXQOt7jwcAQ.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-bMZnCE6bZz7pQvbmsNuung.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-e5CecP2VWJbaDWXFlJISVg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-7D4XhiI1exkz7Xg5slhAyQ.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-qRMOAZg7X3PC4g8XcAEMYg.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-bFHG9wsTT1v2wGc79yiQyA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-65wjSCqIA0n-X8mAtZ8T6w.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-fxNHBq-usS84dy4LrdSPEw.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-KGmVNQ7ZRZN1nI8q274IYA.jar
Apr 27, 2020 6:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-0OA7IQcUW3o9y3IoY4sHIA.jar
Apr 27, 2020 6:55:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-f44d1c7zJrLTm4q3JlMdbQ.jar
Apr 27, 2020 6:55:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-22JYVK85eWUpJokRxHgafw.jar
Apr 27, 2020 6:55:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-22JYVK85eWUpJokRxHgafw.jar
Apr 27, 2020 6:55:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-H5wQ2rHuDlRpLYqfloQW7A.jar
Apr 27, 2020 6:55:03 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-22JYVK85eWUpJokRxHgafw.jar&uploadType=resumable&upload_id=AAANsUmiItMaWOQjNODNBoLepMuHErBxKw7oDKECQ804BV7EzlbblYmU809xCOyxDjxKE6lUu1-lLUGZ0sy7vSLrCeFkn5Om1A.
Apr 27, 2020 6:55:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 27, 2020 6:55:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-22JYVK85eWUpJokRxHgafw.jar
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 27, 2020 6:55:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash V2sf0ufajUAqCk6ReTlPrQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-V2sf0ufajUAqCk6ReTlPrQ.pb
Apr 27, 2020 6:55:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 6:55:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-26_23_55_11-15287333344735137176?project=apache-beam-testing
Apr 27, 2020 6:55:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-26_23_55_11-15287333344735137176
Apr 27, 2020 6:55:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-26_23_55_11-15287333344735137176
Apr 27, 2020 6:55:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T06:55:11.308Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 27, 2020 6:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:14.588Z: Checking permissions granted to controller Service Account.
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:18.488Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.119Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.166Z: Expanding GroupByKey operations into optimizable parts.
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.205Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.301Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.337Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.375Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 27, 2020 6:55:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.403Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 27, 2020 6:55:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.800Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 6:55:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:19.879Z: Starting 5 workers in us-central1-f...
Apr 27, 2020 6:55:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T06:55:34.831Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2020 6:55:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:55:46.521Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 6:56:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:56:06.162Z: Workers have started successfully.
Apr 27, 2020 6:56:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:56:06.195Z: Workers have started successfully.
Apr 27, 2020 6:56:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:56:39.705Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 6:56:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:56:39.892Z: Cleaning up.
Apr 27, 2020 6:56:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:56:39.972Z: Stopping worker pool...
Apr 27, 2020 6:58:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:58:05.185Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 27, 2020 6:58:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T06:58:05.223Z: Worker pool stopped.
Apr 27, 2020 6:58:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-26_23_55_11-15287333344735137176 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 60a4c1c0-d088-41fb-b95e-719cad233c20 and timestamp: 2020-04-27T06:58:12.167000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.099
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 22.036 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 12s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/tx6jf76kn6fls
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #432
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/432/display/redirect?page=changes>
Changes:
[github] Fix invalid formatting specifier in verify
[iemejia] [BEAM-9820] Upgrade Flink 1.9.x to 1.9.3
[github] [BEAM-9819] Relaxes the version range for httplib2 to <0.16.0. (#11522)
------------------------------------------
[...truncated 284.60 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 27, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-iytclP88wwJc-ZNtr3FKwQ.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-cNQ4e95aaN7zVkfZPXL80A.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-9KLiENHE-_2BnH7po_zPYg.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT--kcc0dkm4MKPt_uPqCcVPw.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT--_30ktBrkpAyvhBlzg7pBg.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-6fT2UlzSGURrIuZyeRTsvg.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-DeEpBQIa0cfkcBmR7VaisQ.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-9IkhwMyawVgOn9qfcHcKDQ.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-lgQQILe1f8wj26uJbRagVg.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-UONM_7KqfcvNiE6IVprTzg.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-ZRbVpz6vgMNvJk6mgPsYBQ.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-r8kfHKXq2yqadBXZs6aYdQ.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-KmNQKxvvCbZR9okuP5isag.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-v91oxEcWu8ss_cfpnsDe4g.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-OAH1QQvM7LjGskILyXS5mA.jar
Apr 27, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-paPCMEJxpmG775SG3ckcYg.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-2Ih5LbNSlKHuH4kSuPN2Eg.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-_WJaWWQszqPYRt6HHCwyOQ.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-R13gcuXZkUyVebKCLU02SA.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-EzRgsmwSHGnsBD_cSw-4ug.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-PFV6x5Dyg-vwLklveBSN2w.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-agQkKObERqpodwLwPY6XMQ.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-wV-NGTRBQU-CBaU05FgbLQ.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7PrcVJxqf8AvesPpYhL-fw.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7PrcVJxqf8AvesPpYhL-fw.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ZqpC4Azzt2yxe-L420dhYQ.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-N7b5AfPI6NEy07Fmty1abA.jar
Apr 27, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-KR1FjEwd30yByiz57QQvBw.jar
Apr 27, 2020 12:45:16 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7PrcVJxqf8AvesPpYhL-fw.jar&uploadType=resumable&upload_id=AAANsUlIRhPpu9LBNCmjRr7jq57r3Zutsj3UbQFq_jdQd0SRBUq-9-gVSM8YasJikBLklmhmO52uaatEL9ZDp33xCbU.
Apr 27, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 27, 2020 12:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7PrcVJxqf8AvesPpYhL-fw.jar
Apr 27, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 27, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash rs4YEZ5BHgGhgo_TDcT3aw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rs4YEZ5BHgGhgo_TDcT3aw.pb
Apr 27, 2020 12:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-26_17_45_24-9378169878324709877?project=apache-beam-testing
Apr 27, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-26_17_45_24-9378169878324709877
Apr 27, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-26_17_45_24-9378169878324709877
Apr 27, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T00:45:24.803Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 27, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:27.893Z: Checking permissions granted to controller Service Account.
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:31.932Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.527Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.614Z: Expanding GroupByKey operations into optimizable parts.
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.645Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.734Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.771Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.799Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 27, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:32.833Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 27, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:33.248Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:33.315Z: Starting 5 workers in us-central1-f...
Apr 27, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:45:58.639Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 27, 2020 12:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-27T00:46:07.108Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2020 12:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:46:17.714Z: Workers have started successfully.
Apr 27, 2020 12:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:46:17.740Z: Workers have started successfully.
Apr 27, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:46:51.780Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 27, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:46:51.983Z: Cleaning up.
Apr 27, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:46:52.066Z: Stopping worker pool...
Apr 27, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:48:33.811Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 27, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-27T00:48:33.843Z: Worker pool stopped.
Apr 27, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-26_17_45_24-9378169878324709877 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a3c54195-49b2-42d9-b9fa-36dfb0373c3b and timestamp: 2020-04-27T00:48:40.338000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.517
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 35.756 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 22s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/dtjiilirchz5y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #431
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/431/display/redirect>
Changes:
------------------------------------------
[...truncated 283.11 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 26, 2020 6:45:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 6:45:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 26, 2020 6:45:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 6:45:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 26, 2020 6:45:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 26, 2020 6:45:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 26, 2020 6:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 26, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-PgWoudx6ZQ9LQbBkGL0M4A.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Sfq7OUIz7wTp4AWCF2qAJA.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-gnJWidtp-eWyzEvWDf5EsA.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-ITMYiJt1Kp10LZaLyayVGw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Dp1F7_L8jqFi9hxUPAuhog.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6dkeTB9KPSV2jDNqeP78-w.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Kvp38WXhfktXQ0hpkJaGQw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-DxpTlm2Q22LI4mJDVGyA_g.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-u4vS7G9WU-cdmIbzwwp7hw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-sBlk1OSATNtVb_juxey6Ow.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-zzaUXmtQjOr2ARkLsp2Aug.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-x4RDcrIs4_Hyt3EzpICaZQ.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-7jQ19jZBz_a9Qk_SSVaUjg.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-LK79LEj3B_9wzWAeaRSpJg.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-8uLWK9crUDVVMAakXckspg.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-yN9MFu3i6DDQ8WXfxylSJw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-N4ZIVoiHvo2zI9y2jwx-Rw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-qbo4jI7ooXbaVOSBHujXlw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Wm9fww6TSYsJVyKe6tUCVw.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-eDgSV-KrqngVgirfU1jECA.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-PRfeU1jU3yuNDmBFV4HpJQ.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-ZB1JvH5g7-cYreby07ISWA.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-odJ0dEgmaNIx9eXbWNsn3A.jar
Apr 26, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-SRhY2pFVB97_M89kh96hPg.jar
Apr 26, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-tRTuIBMg9bSroMRoHIlpOQ.jar
Apr 26, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8co5bo1m0NqXr9VNhCspgw.jar
Apr 26, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-DB3BPty3zkhYgJtxvh1N_w.jar
Apr 26, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8co5bo1m0NqXr9VNhCspgw.jar
Apr 26, 2020 6:45:21 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8co5bo1m0NqXr9VNhCspgw.jar&uploadType=resumable&upload_id=AAANsUnl_ajQODgAwOABpWRqSvKQgXaaFsXB7LQ-sxBSJRv-97-U3HIaXIVHW5kVMte-qxeWgEl1s_D8osp2CaDpq74.
Apr 26, 2020 6:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 26, 2020 6:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8co5bo1m0NqXr9VNhCspgw.jar
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 26, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 26, 2020 6:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash tAzMYG2_694W2MxngQK1Uw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-tAzMYG2_694W2MxngQK1Uw.pb
Apr 26, 2020 6:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 26, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-26_11_45_26-7832163380475464135?project=apache-beam-testing
Apr 26, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-26_11_45_26-7832163380475464135
Apr 26, 2020 6:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-26_11_45_26-7832163380475464135
Apr 26, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T18:45:26.587Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 26, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:29.602Z: Checking permissions granted to controller Service Account.
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:33.343Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:33.890Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:33.936Z: Expanding GroupByKey operations into optimizable parts.
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:33.965Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.057Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.090Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.128Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.167Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.704Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 6:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:45:34.788Z: Starting 5 workers in us-central1-f...
Apr 26, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T18:45:42.461Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 26, 2020 6:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:01.491Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 6:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:01.526Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 26, 2020 6:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:06.865Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 6:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:21.334Z: Workers have started successfully.
Apr 26, 2020 6:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:21.364Z: Workers have started successfully.
Apr 26, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:52.406Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:52.619Z: Cleaning up.
Apr 26, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:46:52.702Z: Stopping worker pool...
Apr 26, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:48:36.228Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 26, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T18:48:36.269Z: Worker pool stopped.
Apr 26, 2020 6:48:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-26_11_45_26-7832163380475464135 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 24075234-b0bb-4b3e-a4f5-534c5828c10a and timestamp: 2020-04-26T18:48:42.354000000Z:
Metric: Value:
read_time 14.328
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 33.737 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 24s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/eosxdsfwjy7le
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #430
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/430/display/redirect>
Changes:
------------------------------------------
[...truncated 283.00 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 26, 2020 12:49:47 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-HhhzVpDqDmaT8W92G2QPcA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-VrjY2vNY7UBQmgmp7ZL_rA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-_rPvH_e309Fza5JOQguyxA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-xQjq6WI_P3gmuH3-rkjZVA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-nFYhG4TRrRbR1907TTLBdw.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-S7gEjl26tnJBYsxREGUOAQ.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-o1qMTzeDHV6cay4-leMWjw.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-V1VhMee4YKrM3v5ZHAD-Pw.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-zTu514GoLINOX5Ee-x2IJw.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-3QuIEj6y9pKReFhNmVSOTg.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-m2eYMAh3WiUpWJwcSzFAGQ.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-VrlP7ASlPvev5k1P_I43_Q.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-d6nH6Mgv2xhyWg-jFpaKKA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-QDVA9X2dulg8Z3yRGAhRsg.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-G42WhykGC-GIy0M_ILW-ZA.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-lcKqjvZ3IeJN-V_X6AaIQg.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-2htk-Cn9YuAIyyvakIlfuw.jar
Apr 26, 2020 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-NjnW2GChxEVaO1Qcli8nSQ.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-n1Cb_DML8EEWNLUVeUshWg.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-SCisPMVzNAgtnlHQ5i2xaA.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-eQYtQG13CGr1spN6acRCPg.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-WtbEYZH7mCNsqH3oldBU8A.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-kzcigEKdc8kuM9lRSNXs2g.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-cQXqya4_3g6Gr53Eg2hquQ.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-uXRmf-5kIM_TGMcLuApujA.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-1GOJ0Yb-xGJL-daHBSDJVw.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XWpwYtoPyZ505Gr4lD27Gg.jar
Apr 26, 2020 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XWpwYtoPyZ505Gr4lD27Gg.jar
Apr 26, 2020 12:49:50 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XWpwYtoPyZ505Gr4lD27Gg.jar&uploadType=resumable&upload_id=AAANsUmSh8juHH_m_Y0bposBFwcQIM54KT9fmv-Ro2g_jEYWceaNntPCI0eFgrq_NCSEd9BqItm0bL-iz8jv9uXOJe8.
Apr 26, 2020 12:49:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 26, 2020 12:49:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XWpwYtoPyZ505Gr4lD27Gg.jar
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 26, 2020 12:49:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12106 bytes, hash QDSQB8mXDIlUkbzgvL2C8g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QDSQB8mXDIlUkbzgvL2C8g.pb
Apr 26, 2020 12:49:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 26, 2020 12:49:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-26_05_49_57-6294384414926930057?project=apache-beam-testing
Apr 26, 2020 12:49:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-26_05_49_57-6294384414926930057
Apr 26, 2020 12:49:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-26_05_49_57-6294384414926930057
Apr 26, 2020 12:49:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T12:49:57.196Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 26, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:00.484Z: Checking permissions granted to controller Service Account.
Apr 26, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:04.284Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:04.865Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:04.916Z: Expanding GroupByKey operations into optimizable parts.
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:04.954Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.045Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.076Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.111Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.146Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.605Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:05.694Z: Starting 5 workers in us-central1-f...
Apr 26, 2020 12:50:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T12:50:29.290Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 26, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:31.452Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 12:50:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:31.486Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 26, 2020 12:50:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:36.939Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 12:50:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:50.727Z: Workers have started successfully.
Apr 26, 2020 12:50:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:50:50.762Z: Workers have started successfully.
Apr 26, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:51:27.474Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:51:27.684Z: Cleaning up.
Apr 26, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:51:27.772Z: Stopping worker pool...
Apr 26, 2020 12:53:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:53:31.456Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 26, 2020 12:53:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T12:53:31.511Z: Worker pool stopped.
Apr 26, 2020 12:53:37 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-26_05_49_57-6294384414926930057 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a3b895ea-81b3-4570-bb55-dfbd089b40f8 and timestamp: 2020-04-26T12:53:37.206000000Z:
Metric: Value:
read_time 18.432
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 58.298 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 43s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/dz2iyscswnuvi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #429
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/429/display/redirect?page=changes>
Changes:
[github] [BEAM-9488] Ensure we pass through PCollection ids instead of attempting
------------------------------------------
[...truncated 283.39 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 26, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 26, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 6:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 26, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 26, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 26, 2020 6:45:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 26, 2020 6:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-fAB1QaACnQUdhVvlHHvsIA.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-SDnlKuWf69dlaqnnsO24Ww.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-nn1Y_zhRx3AS32_KX6166A.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Umj0iLYhr_7A3hee2vMR8A.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Xu8S93ah5clKwgkap1UM7w.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-SsmM3FiTV4lbug96VhKhRw.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-hdKOXRxb9MCDe4lHN18r9Q.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-vfM1m5z2sT0IPQEX0Thweg.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-TKnDDnWhbETUsC7Hobd0aw.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-e5zYW1l7jyzti98uFWAfRA.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-LT9IkBecDhQPzHP4pYIAoQ.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-MYZBmHTKCd2LaB_Vyb4VGw.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-3WPtv8NU-i2edeAB9BFcbw.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-H1bZH8hADQhOmuigB0xRNQ.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test--N2Nxv4PLfwn7a5hrTkIPQ.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-a1xlJJdLXUZumQBx7SNXHA.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-SO1yp84pv59636pggvaKGg.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-diPp-65ksLZbCo5tGpiyCg.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-6662mm3gPpNTzrt-_xYocQ.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-JVj0-9iILJDpo1Ar47aN3w.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-hTfqQtjG492v8nvnaIha4w.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-EXnVPuPKO3UrSWbOw4uZBA.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-1gYirtubqQHQwGDWzIlCGg.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-I5-pLNx-7xeFMxFxYyiKrQ.jar
Apr 26, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-LP__i8ooPenPt9W-SQnoTw.jar
Apr 26, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-nQLNuIIOsV_-kB6vHSH3dQ.jar
Apr 26, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-UveM16S8aGvcCSZGKH0cAQ.jar
Apr 26, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-I5-pLNx-7xeFMxFxYyiKrQ.jar
Apr 26, 2020 6:45:17 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-I5-pLNx-7xeFMxFxYyiKrQ.jar&uploadType=resumable&upload_id=AAANsUk2Gxh83iC9sOtyPwnLIdJYfN3aoXDNSTPHS_Z-a8P94bV1ze5yc43tiM86HhYetTnFzo3T7LXs8X669Q1q-rM.
Apr 26, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 26, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-I5-pLNx-7xeFMxFxYyiKrQ.jar
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 5 seconds
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 26, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1JoUM5wxYqZxAQ7aFYFxrw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1JoUM5wxYqZxAQ7aFYFxrw.pb
Apr 26, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 26, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-25_23_45_22-12209034000419486336?project=apache-beam-testing
Apr 26, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-25_23_45_22-12209034000419486336
Apr 26, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-25_23_45_22-12209034000419486336
Apr 26, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T06:45:22.374Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 26, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:25.517Z: Checking permissions granted to controller Service Account.
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:29.979Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.568Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.708Z: Expanding GroupByKey operations into optimizable parts.
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.736Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.801Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.833Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.858Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:30.898Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:31.359Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:31.441Z: Starting 5 workers in us-central1-f...
Apr 26, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T06:45:49.893Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 26, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:58.088Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:45:58.122Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 26, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:03.525Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:17.294Z: Workers have started successfully.
Apr 26, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:17.329Z: Workers have started successfully.
Apr 26, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:59.210Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:59.416Z: Cleaning up.
Apr 26, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:46:59.504Z: Stopping worker pool...
Apr 26, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:48:39.512Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 26, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T06:48:39.556Z: Worker pool stopped.
Apr 26, 2020 6:48:46 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-25_23_45_22-12209034000419486336 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 87840f80-dba0-4870-be74-eb8e276b3eab and timestamp: 2020-04-26T06:48:46.059000000Z:
Metric: Value:
read_time 21.815
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 39.768 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 28s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/cc2fxputptw3q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #428
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/428/display/redirect>
Changes:
------------------------------------------
[...truncated 283.44 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 26, 2020 12:45:11 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 26, 2020 12:45:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 26, 2020 12:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-o7JjLNBDWuvMti7P-rFi7A.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-OY6SOGp2Zeqza8TL1CvoFA.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-EaeJ35rroIOU_SDxm0pZ_g.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-2qItS3YtlztARHDz6Q-odQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-NbcpEacHfiw9YaCKmaOUJg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-vaKjjMV_SolBIQwaRprE7A.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-1tLBZu35CDWAAOFrukIzMg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-X1JAKgr-Ck4xVCr3X0G9zg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-iDeniYtGcW5zTNWfDN1Mcg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-gpwml-iXIcbXqiBzVzo9kQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-GWY_wA2vTJ1I8OkEmjpSPw.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-tVMxh-zR02W-9K8Bee3sFw.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-dzA_fH_Na5THYAyBgdAgmQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-s7Zq4VNZg8tAM2lqkyAZcQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-07wnllXlAe830J0ke2xAxg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-DCAxiTag6p-EpLjPk6hH7Q.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-snw_cge0JrO1V4Gva46S2Q.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-gtzW5IyRu0rLtTeRdP-zKQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-SRyKnzweLwTRqtznTrpnzg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-fiqXMPiGpTj8ajRQpcbvzg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-onADtElk38nw0SXBjUG4uA.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-oN_z03eb59NCc3bHfluwWg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-J5-JiRk2AAj7m7o-85UkUg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZJ073MlmuHPxppjv7g5RqA.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-xMG7k5PIfsLNml2csZT_pQ.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-nJ9qVAgumRTwjVpEiLYwVg.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-e1kFnJu0gUcgDEwY0C0-rw.jar
Apr 26, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZJ073MlmuHPxppjv7g5RqA.jar
Apr 26, 2020 12:45:15 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZJ073MlmuHPxppjv7g5RqA.jar&uploadType=resumable&upload_id=AAANsUlRe-I3bjMhUxfzSB9Io7cfu5bj8nK5xKyVDI-Vn-xT20sW7Bp43eBbqwZoOWUpGDRHfg4Uqrii4siWHA_Y7sfrbeczHw.
Apr 26, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 26, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZJ073MlmuHPxppjv7g5RqA.jar
Apr 26, 2020 12:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 26, 2020 12:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 26, 2020 12:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 26, 2020 12:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 26, 2020 12:45:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 26, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 26, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash _ZhKJdQnd_K-fy5u-LjFSQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_ZhKJdQnd_K-fy5u-LjFSQ.pb
Apr 26, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 26, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-25_17_45_23-1774983237330137963?project=apache-beam-testing
Apr 26, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-25_17_45_23-1774983237330137963
Apr 26, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-25_17_45_23-1774983237330137963
Apr 26, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T00:45:23.752Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 26, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:27.208Z: Checking permissions granted to controller Service Account.
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:31.374Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:31.996Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.036Z: Expanding GroupByKey operations into optimizable parts.
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.061Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.130Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.163Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.199Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 26, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.224Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 26, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.545Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:45:32.617Z: Starting 5 workers in us-central1-f...
Apr 26, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-26T00:46:00.602Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 26, 2020 12:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:46:03.276Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 12:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:46:03.305Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 26, 2020 12:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:46:08.681Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 26, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:46:24.332Z: Workers have started successfully.
Apr 26, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:46:24.362Z: Workers have started successfully.
Apr 26, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:47:01.662Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 26, 2020 12:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:47:01.863Z: Cleaning up.
Apr 26, 2020 12:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:47:01.937Z: Stopping worker pool...
Apr 26, 2020 12:48:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:48:23.949Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 26, 2020 12:48:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-26T00:48:23.998Z: Worker pool stopped.
Apr 26, 2020 12:48:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-25_17_45_23-1774983237330137963 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 13a2e28f-263b-484e-9df3-f4c4a2825a0f and timestamp: 2020-04-26T00:48:31.021000000Z:
Metric: Value:
read_time 18.906
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 28.478 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 13s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/fzgeer5yspsrs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #427
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/427/display/redirect>
Changes:
------------------------------------------
[...truncated 284.53 KB...]
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 25, 2020 8:11:48 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 25, 2020 8:11:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 25, 2020 8:11:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-EUROn7tj8W2wEA0Bi0v0-g.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-YHjwLsBfs-p12uRlZgeItg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-StGyRPSpwm8V0S-xm1ok4w.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-FZ__2nWXAkiutvfCeen1Kg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-MOrS_tKpzUU0q7rnTTE1Ww.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-FP5LapsngTu5AQJLISZvaA.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-qf0jZ6FNPwKnCH_qh7UG_w.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-TxThbDVdSfsliQfaOeuhwg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-isKDSjBGXyH4AjWjoNVfeg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-_UdToiCnnxwbdwmsC1WqFw.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-4sXYwBdDq0U_arjHTir0zw.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6K2iRb73Hs1d1RMSUsoQ3A.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-BSCPbvp3ZXDIGuycGINa-w.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-uvm4TtkHS270BNaPRkM3ug.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ddMlXaItajfDAI5jyf0r2A.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-ZNxmFH4VTLcw1-w2bhltpA.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-Z0Gexz_jik0MMeK0wpyHFQ.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-nVrPSk4Ac5Ey5Hh-wsfC3A.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-VqxOhr5TPaEIFKD0ixrazQ.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-WcrtatJDQOoPFwr6n0Dhgg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-D-NAxMg140PTI0Y35GKL9g.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-csv-1.8-kBnZnWBy9I_ZsfkJ8MRdJA.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-TIAYxHV55iJONt_ZOnLEwg.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-nal__Tr3DNtAD2sO0SBpMw.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-PtVGvWN2LpLu3itEU1DwZQ.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-rNSCBCyaIURKad_Lo_tECw.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GkVFcx-jOGakH03NEtMFaw.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-IoD9vSBp6kzBPqWr42-m6w.jar
Apr 25, 2020 8:11:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GkVFcx-jOGakH03NEtMFaw.jar
Apr 25, 2020 8:11:52 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GkVFcx-jOGakH03NEtMFaw.jar&uploadType=resumable&upload_id=AAANsUlXgH7TxYnPYPKA0T6uNzn1cFgVLGWwpAFyBemE9X_TpmZORdo4iFWrg9jgrVDCpwytbHaSxfqpbFPM1TD9VsZmSfXuEQ.
Apr 25, 2020 8:11:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 25, 2020 8:11:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-GkVFcx-jOGakH03NEtMFaw.jar
Apr 25, 2020 8:11:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 29 files newly uploaded in 9 seconds
Apr 25, 2020 8:11:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 25, 2020 8:11:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 25, 2020 8:11:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 25, 2020 8:11:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 25, 2020 8:11:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 25, 2020 8:11:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash AeoQ7fdksMYBgGVGP5LwgA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-AeoQ7fdksMYBgGVGP5LwgA.pb
Apr 25, 2020 8:11:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 25, 2020 8:12:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-25_13_11_59-8748211962317737408?project=apache-beam-testing
Apr 25, 2020 8:12:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-25_13_11_59-8748211962317737408
Apr 25, 2020 8:12:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-25_13_11_59-8748211962317737408
Apr 25, 2020 8:12:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T20:11:59.686Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 25, 2020 8:12:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:02.505Z: Checking permissions granted to controller Service Account.
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:06.192Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:06.922Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:06.949Z: Expanding GroupByKey operations into optimizable parts.
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:06.975Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.043Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.073Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.095Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.136Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.581Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 8:12:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:07.658Z: Starting 5 workers in us-central1-f...
Apr 25, 2020 8:12:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T20:12:26.091Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 25, 2020 8:12:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:34.674Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 8:12:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:34.710Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 25, 2020 8:12:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:40.206Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 8:12:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:53.235Z: Workers have started successfully.
Apr 25, 2020 8:12:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:12:53.269Z: Workers have started successfully.
Apr 25, 2020 8:13:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:13:26.555Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 8:13:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:13:26.768Z: Cleaning up.
Apr 25, 2020 8:13:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:13:26.868Z: Stopping worker pool...
Apr 25, 2020 8:15:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:15:18.162Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 25, 2020 8:15:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T20:15:18.206Z: Worker pool stopped.
Apr 25, 2020 8:15:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-25_13_11_59-8748211962317737408 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 44998a5d-3aea-404a-8d2d-89d908a555f2 and timestamp: 2020-04-25T20:15:24.081000000Z:
Metric: Value:
read_time 14.984
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 44.789 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 51s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7dkn4jyrr6slq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #426
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/426/display/redirect>
Changes:
------------------------------------------
[...truncated 283.20 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 25, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-toy2IwnTsXIT53iTuwL-3A.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-YuS8IjgpF1O637hBwqx2Jw.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-0PpRT3W6_nMQjvQjlA-hIw.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-t0j2gMICZJycSt7GnvHYfA.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-gZnfg01fVvDYjPkBWnhXDw.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-DDa3KKBnjZJEJ_GwKR126Q.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-zf3Fcru-ozPxW7z5inlzEw.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-ZoNbtVNCq21ZaByPfF5ijQ.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-GIRg5vwE4B2yUOvc3nh-sQ.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-IEn5SLNY7aIMMIT7-Yy7aQ.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-_U40QfckDwrwzkOqrfwi1w.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Vjbn6AT9138lGzzeZAIXPg.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-YLYOtjXEQQ63hj4UamKzbA.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-NbFxsDIv3cotYen1R35Wzw.jar
Apr 25, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-mh0uF7LhKnzS-4EvitTmeA.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-sK5aUVxCEjug9V33bqyVeg.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-DaC8lwdITQo6216T-V7jRw.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-g8oX3bvtfJUVUzBX8TTKKA.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Fzuxma6mJ05ihonPzOS_3g.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-eiGvbN2FSfUn-jRYlPGqoQ.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-_naBazKxu_4VH9dvu5rwDw.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Jnpu7guxJc_HSAk2AXBNFg.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-LoKLT003ymKsmlxrabeFyQ.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-3pGyOC3Iz37q9XlZ0QqtCA.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-FszeREyFncW0VNbv6jYsUg.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-SbekfZgr0THyYMddyNQn2w.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZlWDyX38o3fS3O_Fe4NhPA.jar
Apr 25, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZlWDyX38o3fS3O_Fe4NhPA.jar
Apr 25, 2020 6:45:33 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZlWDyX38o3fS3O_Fe4NhPA.jar&uploadType=resumable&upload_id=AAANsUknuENhFOf8IcEOX3VLVkF4VpNCnVLVdTQ3mCtuU4IOVd4v--TqysFVucUiFukXpADWH5IrYeGyOvxgBUdN_XE.
Apr 25, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 25, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZlWDyX38o3fS3O_Fe4NhPA.jar
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 25, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 25, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash GjlXqt2SiP51HDjJ8f9ayA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-GjlXqt2SiP51HDjJ8f9ayA.pb
Apr 25, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 25, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-24_23_45_41-4829189551288583944?project=apache-beam-testing
Apr 25, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-24_23_45_41-4829189551288583944
Apr 25, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-24_23_45_41-4829189551288583944
Apr 25, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T06:45:41.515Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 25, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:44.943Z: Checking permissions granted to controller Service Account.
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.073Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.689Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.729Z: Expanding GroupByKey operations into optimizable parts.
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.759Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.829Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 25, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.851Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 25, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.881Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 25, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:48.906Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 25, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:49.185Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:45:49.256Z: Starting 5 workers in us-central1-f...
Apr 25, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:46:15.246Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:46:15.279Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 25, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T06:46:19.037Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 25, 2020 6:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:46:20.716Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:46:34.488Z: Workers have started successfully.
Apr 25, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:46:34.517Z: Workers have started successfully.
Apr 25, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:47:12.470Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:47:12.655Z: Cleaning up.
Apr 25, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:47:12.727Z: Stopping worker pool...
Apr 25, 2020 6:48:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:48:39.202Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 25, 2020 6:48:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T06:48:39.261Z: Worker pool stopped.
Apr 25, 2020 6:48:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-24_23_45_41-4829189551288583944 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 126c6396-9a30-4478-9646-a687d226a713 and timestamp: 2020-04-25T06:48:45.835000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.363
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.042 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 24.575 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 28s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/ojkdq7lhhzwmu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #425
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/425/display/redirect>
Changes:
------------------------------------------
[...truncated 283.70 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 25, 2020 1:06:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 1:06:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 25, 2020 1:06:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 25, 2020 1:06:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 25, 2020 1:06:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 25, 2020 1:06:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 25, 2020 1:06:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 25, 2020 1:06:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-YNt7OvQx8kWnJ9_pjzqyTg.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-F4OQCxeNitxPgvenbm_LuA.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-QTFmbJ9iGQeeT0F5l7o1rw.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-wobij2uXiNgDdcQqxa_g1g.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-dZNyORi7r-NhGDqFALidYQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-iitsvlO6y49CaZiBpmBzAQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-qO49H9tc7eeKJRnBoFLBBQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-XeJUU-Cej93cQODiEjfvEQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Bm7OaCsIdJMZrLtFQY2xqQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-DjOL70uz-NwbmphgMb-cxw.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-P4bW5rVUmc3J2KfILcRFaw.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-cgrdsCwrb_RG9a54hyR4qQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-IgiQt2Dk_SDt8X7LtoG4Vw.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-gMGzdgIT_-9UDaq_KW353g.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-tYZHied02eHWqccTQRI53w.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-gjBitqQHPiLvEG6l_8fshQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-OWAN5VCw_hynx5biU6Hgww.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-FMiMk99WnkOJ6IVJPej1jg.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-iADLJynrBBeDvL9av0GLrw.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-bosmM2BLeqjgc2IboReSjQ.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-EiU6DWYJHZnWKbL2YxBFRg.jar
Apr 25, 2020 1:06:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-bDgSLvSsUlAooc8Q_WRXVQ.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-RkyX3RtvKrt1yRh6lOXAsw.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-7o6HlHCUxzhZNnPZtzNC_w.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-XCdzesvmjinPJebYWCCebw.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1ah7RS22f-TbxDqnW0BzbQ.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-2_8Azmj-4j39XXcMIjgEUA.jar
Apr 25, 2020 1:06:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1ah7RS22f-TbxDqnW0BzbQ.jar
Apr 25, 2020 1:06:25 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1ah7RS22f-TbxDqnW0BzbQ.jar&uploadType=resumable&upload_id=AAANsUnYPGIxpTmtSldxF6bopzeMM-Ipg3NjDiaePj96-xIUqM9QG9wYaxPIz0_XnGmKNg7_OnqIz5b7WQS06Ybcdso.
Apr 25, 2020 1:06:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 25, 2020 1:06:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1ah7RS22f-TbxDqnW0BzbQ.jar
Apr 25, 2020 1:06:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 25, 2020 1:06:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash hbiMGA4lPsyjy9JcE6D4Zg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-hbiMGA4lPsyjy9JcE6D4Zg.pb
Apr 25, 2020 1:06:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 25, 2020 1:06:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-24_18_06_35-5061305351598070795?project=apache-beam-testing
Apr 25, 2020 1:06:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-24_18_06_35-5061305351598070795
Apr 25, 2020 1:06:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-24_18_06_35-5061305351598070795
Apr 25, 2020 1:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T01:06:35.090Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 25, 2020 1:06:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:38.694Z: Checking permissions granted to controller Service Account.
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:42.526Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.211Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.261Z: Expanding GroupByKey operations into optimizable parts.
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.296Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.394Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.429Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.469Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:43.501Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:44.139Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 1:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:06:44.234Z: Starting 5 workers in us-central1-f...
Apr 25, 2020 1:06:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-25T01:06:50.769Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 25, 2020 1:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:08.371Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 1:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:08.401Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 25, 2020 1:07:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:13.858Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 1:07:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:13.900Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 25, 2020 1:07:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:33.006Z: Workers have started successfully.
Apr 25, 2020 1:07:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:33.034Z: Workers have started successfully.
Apr 25, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:07:51.601Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 25, 2020 1:08:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:08:08.915Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 25, 2020 1:08:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:08:09.113Z: Cleaning up.
Apr 25, 2020 1:08:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:08:09.184Z: Stopping worker pool...
Apr 25, 2020 1:10:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:10:07.881Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 25, 2020 1:10:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-25T01:10:07.944Z: Worker pool stopped.
Apr 25, 2020 1:10:13 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-24_18_06_35-5061305351598070795 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ff9ac738-ad26-4d77-a3b5-46e7613b022a and timestamp: 2020-04-25T01:10:13.831000000Z:
Metric: Value:
read_time 17.287
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 4 mins 0.551 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/hm4aecg7duftm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #424
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/424/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9813] Replace outputs with respect to the correct tag.
[kamil.wasilewski] [BEAM-9761] Disable auto builds for tests that are currently failing.
[tweise] [BEAM-9811] Copy pom for all publishMavenJavaPublication tasks
[github] add licenses and license types for zetasql packages (#11516)
[github] Added a batch example with covid tracking data for interactive notebook.
------------------------------------------
[...truncated 283.44 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 24, 2020 6:50:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 6:50:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 24, 2020 6:50:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 6:50:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 24, 2020 6:50:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 24, 2020 6:50:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 24, 2020 6:50:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 24, 2020 6:50:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-hRcIHPNAuPI5VWU8Pk2wEw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-PUVpLB_235xkiMjJP-33iw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT--SM7113uEZ25vd047eQTSg.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-bwO7RoRzir4orqbudCgQ1w.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-aS1Nu3olVQTnkNsk6MJspg.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-k-jTz_vENAw88seSAIzoZQ.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-JOzCsrqRCYBkJi4A0GXEmg.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-T7D7GP98k0EwylP92To2GA.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-lv1PvBqij2dRSbJ6cljrXA.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-8ZI0WLCCBcXZwB7GFOKo3A.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-T5DunmU5de6SpVcF5F27dg.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-XwnD2X1Z0rIgkNk8jasQOQ.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-0E-1KwEMHgStBFAW3_743w.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-XcSdQ_SgCu65iDgulAOTIQ.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-JKKtnPL49rdA5NBtYdQDPA.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-O2orW4jJpYMntUkMP7VRHw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-sYaLBw6aMSH3KDM9-tcsGw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-5s4Tx7R0RN7E_xspOdd0uw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-wumpJWaoHXVxlBXZGAaFNQ.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-hSagdezHaLoigV9zEYsGfQ.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-NHPapzrlTJepV0lhP_-M1w.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-kVlJmWVWySIVAphOJX6t_Q.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-nBuzIp9ibExYwY3DjL5pCw.jar
Apr 24, 2020 6:50:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-qrYxHliTKlRV7_9RBrmArg.jar
Apr 24, 2020 6:50:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gTF_ckJlijchwWLGbkUfvQ.jar
Apr 24, 2020 6:50:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-lpS1KJ_5BqRxF7bornN7Ow.jar
Apr 24, 2020 6:50:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bfoJCIN0qGgh306tPZu4mg.jar
Apr 24, 2020 6:50:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bfoJCIN0qGgh306tPZu4mg.jar
Apr 24, 2020 6:50:30 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bfoJCIN0qGgh306tPZu4mg.jar&uploadType=resumable&upload_id=AAANsUk4-rxdqTCdEMyO_piKciTxoyv-4eP5CWux_eVwcouoWonCYd59jAuoPpGagzxyI0bIw1klF88STEztnulCE1Fn23G4ew.
Apr 24, 2020 6:50:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 24, 2020 6:50:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bfoJCIN0qGgh306tPZu4mg.jar
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 24, 2020 6:50:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 24, 2020 6:50:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash w_uK170Z9mu5bJKSQtXXyQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-w_uK170Z9mu5bJKSQtXXyQ.pb
Apr 24, 2020 6:50:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 24, 2020 6:50:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-24_11_50_35-4774656204841605778?project=apache-beam-testing
Apr 24, 2020 6:50:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-24_11_50_35-4774656204841605778
Apr 24, 2020 6:50:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-24_11_50_35-4774656204841605778
Apr 24, 2020 6:50:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T18:50:35.657Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 24, 2020 6:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:39.818Z: Checking permissions granted to controller Service Account.
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:45.921Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:46.650Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:46.777Z: Expanding GroupByKey operations into optimizable parts.
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:46.845Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:46.996Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:47.048Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:47.097Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 24, 2020 6:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:47.141Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 24, 2020 6:50:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:47.644Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 6:50:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:50:47.762Z: Starting 5 workers in us-central1-f...
Apr 24, 2020 6:51:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T18:51:18.812Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 24, 2020 6:51:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:51:22.996Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 24, 2020 6:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:51:38.925Z: Workers have started successfully.
Apr 24, 2020 6:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:51:38.968Z: Workers have started successfully.
Apr 24, 2020 6:52:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:52:15.219Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 6:52:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:52:15.536Z: Cleaning up.
Apr 24, 2020 6:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:52:15.687Z: Stopping worker pool...
Apr 24, 2020 6:54:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:54:26.100Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 24, 2020 6:54:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T18:54:26.183Z: Worker pool stopped.
Apr 24, 2020 6:54:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-24_11_50_35-4774656204841605778 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 047219e8-2dd0-4ce4-aa2c-41e181c4e565 and timestamp: 2020-04-24T18:54:35.014000000Z:
Metric: Value:
read_time 19.007
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 4 mins 16.807 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 22s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/aki2oa4osru3m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #423
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/423/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9733] Always let ImpulseSourceFunction emit a final watermark
[mxm] [BEAM-9733] Make up for timers set while processing the bundle
------------------------------------------
[...truncated 284.16 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 24, 2020 12:48:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 24, 2020 12:48:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 24, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-XTi87BtwlJstgZRsIevIJA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-LX2pkfBU_r40fqx93BDcVA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--ATXMRZNDPabKUpF1nkkKQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-t4BcfeDOczatnl5n-1jtMQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aYcgLLMp2FzltOMJCl-SZg.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-bA7kZGRxD1mAZR8mc3CfBw.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-AQrKLnDel0jhvjTfLuNJNg.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-RF3_KOAlp5yFwCxaLgbRCA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-cxHrwSryFWcOYN-gF5u6WA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-toHrsjQRCgkZWpuD1Na_lQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-g2RkfrZaSeKnTaJy3hsc9A.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-RnsyEYiFx9VDeQDj5THaqw.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-YWbk1iawyqk74_bEyDFXuA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-B-eUXrt9TWD9TdreAVlJoA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-433QxJOLBb_slXvsgnsHeQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-aMTZQTL8nZ-ClC6tvaoENQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-cZm8cixMyHoSFvIs5-UuWQ.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-UdhNVKPqHJEwll6jC7MNWg.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ZMgtHae_nRuUd_rcF3Y5fw.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Zcq1IlCyTiQMIFzn_5FrXA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-DzAPOce4ZIMWqG9lVgA3CA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-07blpIkyY1sQgPAaziC2fA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Z25Ls3u5v1rUL-GkYqilFg.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-_afPL4i0FajmNYVABDWZIA.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-M4xSk2xz5oYhM2H8IR3LBw.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-iEHdsq2y1q7Y4umwoO36Nw.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Z25Ls3u5v1rUL-GkYqilFg.jar
Apr 24, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-WdOGoUw1zw8MS78URT6CJA.jar
Apr 24, 2020 12:49:01 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Z25Ls3u5v1rUL-GkYqilFg.jar&uploadType=resumable&upload_id=AAANsUmLVhYdCbC063It_lWIOsey4ykkD07uC_MswVWEmgdT7-lRWEng7uHkKqaLQvGmHa7pJgPRWdMJjHaa_1JPC4Z-T0hg2Q.
Apr 24, 2020 12:49:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 24, 2020 12:49:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Z25Ls3u5v1rUL-GkYqilFg.jar
Apr 24, 2020 12:49:08 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 24, 2020 12:49:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12097 bytes, hash pxZul6dkMXIzDPENjw0H0A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pxZul6dkMXIzDPENjw0H0A.pb
Apr 24, 2020 12:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 24, 2020 12:49:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-24_05_49_09-4673450521764091760?project=apache-beam-testing
Apr 24, 2020 12:49:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-24_05_49_09-4673450521764091760
Apr 24, 2020 12:49:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-24_05_49_09-4673450521764091760
Apr 24, 2020 12:49:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T12:49:09.836Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 24, 2020 12:49:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:13.877Z: Checking permissions granted to controller Service Account.
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:18.529Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.229Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.262Z: Expanding GroupByKey operations into optimizable parts.
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.297Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.425Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.456Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.484Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 24, 2020 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:19.509Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 24, 2020 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:20.272Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:20.358Z: Starting 5 workers in us-central1-f...
Apr 24, 2020 12:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T12:49:39.686Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 24, 2020 12:49:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:49:45.782Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 24, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:50:03.769Z: Workers have started successfully.
Apr 24, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:50:03.800Z: Workers have started successfully.
Apr 24, 2020 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:50:39.296Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:50:39.488Z: Cleaning up.
Apr 24, 2020 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:50:39.554Z: Stopping worker pool...
Apr 24, 2020 12:52:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:52:17.574Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 24, 2020 12:52:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T12:52:17.613Z: Worker pool stopped.
Apr 24, 2020 12:52:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-24_05_49_09-4673450521764091760 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 990d61a3-57fa-4dd0-ac04-26cd9b388eb6 and timestamp: 2020-04-24T12:52:25.256000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.881
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 35.968 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/saz2p4urcyjru
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #422
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/422/display/redirect?page=changes>
Changes:
[boyuanz] [BEAM-8871] Support trySplit for ByteKeyRangeTracker
[tweise] [BEAM-9811] Nightly snapshot publish error
[github] [BEAM-9775] Adding Go SDF example, adjusting GetProgress signature.
------------------------------------------
[...truncated 283.82 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 24, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 24, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 24, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-D15m6XSF2D0g2PHqECsBtw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-q6on3Bf250tpdppZmsIdEA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-NtsJOPn9NukvKpvIvxmeQA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-vR2Q6xascKYkpMXdelVUAg.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-DLae6Km4naApdr3FGUW4YQ.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-XPP2F6FCikVf9tMi6tvY4w.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-mcsLnnt7sDljFgiTg3ZV2A.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-KagiaaP7RpnOjFpWq_dDJQ.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-l7cPy5vGhrXxIYUlVJP7Bw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-BA9RJYgAsuF-TGE6pnjrEA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-0t9Nbgzr8WUpwc10hn1B4Q.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-DciZTLfLRVsNXOCeVl8Xmw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-0nYPB5jUdgkxuKE6qVeNYg.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-WmpYZ3vfrLj4GaCNvxkMzQ.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-w-4nqUppSK-rbU7LEN4_iw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-uYPPMBDJw9sSYM9MMnsuRA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests--h7PZuY9E1pfmB2H8zfQBA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-c6y5WuIU-X74vydGpy4kjw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-jNByyW8apJ_kjq7vMVCOsA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-gzUBhTKZY0yb0WlcQoYPbg.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-dZ6WmRqoRgwSk0KAD_7x1g.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-K4i1mDg8-thfvZ86ahcorQ.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-VnqGTkCh8kXeJGjL1l1PfA.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-kp2dlmxUng5HA0Hn5nDHHg.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-V0oZUakMrvLyPftj5mrRcg.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-GKIvcu0WZQ3JTSynCMCAXw.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YKgy8B_1njmbZM-E-qgQCQ.jar
Apr 24, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YKgy8B_1njmbZM-E-qgQCQ.jar
Apr 24, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YKgy8B_1njmbZM-E-qgQCQ.jar&uploadType=resumable&upload_id=AAANsUnZHBi2cI0dXbjIQE5SJVBME9_9sEv3VIASQxMMuwgKc0v18pODY4r4re9WAnchQixq40mspXG7oM_9vwQLOS0.
Apr 24, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 24, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YKgy8B_1njmbZM-E-qgQCQ.jar
Apr 24, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash _qWjMlyw2khQhvWBOMzulw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_qWjMlyw2khQhvWBOMzulw.pb
Apr 24, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 24, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-23_23_45_38-13004494617364743648?project=apache-beam-testing
Apr 24, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-23_23_45_38-13004494617364743648
Apr 24, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-23_23_45_38-13004494617364743648
Apr 24, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T06:45:39.011Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 24, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:43.411Z: Checking permissions granted to controller Service Account.
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:47.697Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.383Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.471Z: Expanding GroupByKey operations into optimizable parts.
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.520Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.693Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 24, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.736Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 24, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.774Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 24, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:48.818Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 24, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:49.433Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:45:49.543Z: Starting 5 workers in us-central1-f...
Apr 24, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:46:18.329Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 24, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:46:18.369Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 24, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T06:46:19.303Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 24, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:46:23.809Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 24, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:46:39.300Z: Workers have started successfully.
Apr 24, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:46:39.335Z: Workers have started successfully.
Apr 24, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:47:11.956Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:47:12.327Z: Cleaning up.
Apr 24, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:47:12.484Z: Stopping worker pool...
Apr 24, 2020 6:48:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:48:55.197Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 24, 2020 6:48:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T06:48:55.252Z: Worker pool stopped.
Apr 24, 2020 6:49:01 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-23_23_45_38-13004494617364743648 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2303b69d-497f-46d4-90f9-e43ba6d06f24 and timestamp: 2020-04-24T06:49:01.351000000Z:
Metric: Value:
read_time 13.885
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 41.173 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/usm3v6idqcmei
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #421
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/421/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9577] New artifact staging and retrieval service for Java.
[robertwb] [BEAM-9577] Multi-threaded artifact staging service backend.
[apilloud] [BEAM-9514] Ensure nullability passes through sum
[lcwik] [BEAM-2939] Expose HasProgress interface for restriction trackers and
[lcwik] [BEAM-2939] Add the ability for SDK harness runners to provide
[lcwik] [BEAM-2939] Integrate progress reporting for splittable dofns.
[lcwik] [BEAM-2939] Fold Sizes sub-interfaces into RestrictionTracker
[lcwik] [BEAM-2939] Drop HasSize in favor of using
[robertwb] Use futures, better error handling.
[pabloem] [BEAM-9812] Fixing bug causing pipelines requiring temp tables to not
[ehudm] [BEAM-7405] Workaround for bad Docker config
------------------------------------------
[...truncated 287.05 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 24, 2020 1:05:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 24, 2020 1:05:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-_cMp4BDyoaLYmCPpWWLOIA.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-6MhypqeOTTj55y0VP2PeNQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-heBsszeVUKpIpBjdvXAP5w.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-pGTZDIWtP9eCN2q6kX7g6A.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-YKf9mRf8nfLwNxGU5KV0lQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ESpLeaB9R159lVq6RSx53A.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-xgNr2cvCdkrqmvCzSLT8TQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-gebHQWdEzlpTljmZ4VNtYw.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-t8juaCoNtRcg4MD1vx9B-A.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-DIevnsacV3nr-UiCAK-Hyw.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-vGoKMie5jO_MzYI9k1LoZQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-iG9M-EIhCR9HNvw3HWGm3A.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-sL4u7UXZErQ0NhUnXwm3NQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-rfy7BIKP07ZWokuzD56TaQ.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-oplo6hqJef2UGtrZMfE1yg.jar
Apr 24, 2020 1:05:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tAgeg4Mrr7iKCFR2c5WqMQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-jMo6EqTguYgyZXqqbe_Mgg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Wc42Td9PV4plZvG33N8VvA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-JKuR0v-6yB6Tk-40W-yfjQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Dr2sZ0XId7XZ_c9ImPYU4Q.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-DCprEejB8EhaUHomUiTmcA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-2gJ6p4FjMotCkIYvMV9qgA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.mongo/2.2.0/781d14f4e3d9eeb0b4c3e00a4ec165a04b3dc5c4/de.flapdoodle.embed.mongo-2.2.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.mongo-2.2.0-3bAjs_S7MZ7-neWHp_CgmQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-XUKyFLqYSXZStVFmxkwdbg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fastjson-1.2.49-z0P0MjduZT1wQpv9Ia-IMA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-UdJgsnmZ2Eal05jDeHI4kA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-WP1E8icGATuH7aYb2w3ehw.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-compiler-3.0.11-bepx6Dd9e13zaxGldAayYQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/janino-3.0.11-pubTQ_GoxcNrGgSD5_X7qw.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oUNFxANvu4MDqG0MCASSgg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-4.7-COwoTCR4e1f3u2wy6oa_VA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.process/2.1.2/986b38302fa10018d5baccee7f655c31ee9afe5b/de.flapdoodle.embed.process-2.1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.process-2.1.2-YrD3oe9Ff_itNus4wP3waQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT--7xhAd8V7w0bws8z-eFzsA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oUNFxANvu4MDqG0MCASSgg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ST4-4.0.8-ozr8Njx2YI3X-Kcj8Tfb8Q.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/org.abego.treelayout.core-1.0.3-nIzvq2NgpnJWU3DVMR8PPA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/javax.json-1.0.4-Vphw-XXe7rZpH8ubwCqVVQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr-runtime-3.5.2-H7uuLLclMCB8ILeXvavQKQ.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-runtime-4.7-t59VAkIGs5viU54ez94MCg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-platform-4.0.0-6WVyLGquTreCjQNLObQw0g.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-calcite-1_20_0-0.1-bAqPbFlwv7RdbvHyrAw_yg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/mongo-java-driver-3.9.1-SIrJ2zP3qz8Zl5iCpu_6gg.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-4.0.0-oeIOSKNnBjAj25E3zrfGPA.jar
Apr 24, 2020 1:05:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/icu4j-58.2-YF2KAnaigP9jMsO9JgcRgA.jar
Apr 24, 2020 1:05:43 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oUNFxANvu4MDqG0MCASSgg.jar&uploadType=resumable&upload_id=AAANsUkaUzL1dsjlgg9FIuA0-MCZA6BpBZsu9ZWQ58CYFaxxUHa4FDYp75PgG__x0mnW2FwPwuBtcJc-iHLaNFctEEA.
Apr 24, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 24, 2020 1:05:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oUNFxANvu4MDqG0MCASSgg.jar
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 163 files cached, 44 files newly uploaded in 9 seconds
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 24, 2020 1:05:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 24, 2020 1:05:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash SdIEdXoQfGtK2a8DShMc3g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SdIEdXoQfGtK2a8DShMc3g.pb
Apr 24, 2020 1:05:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 24, 2020 1:05:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-23_18_05_50-6203930417701657601?project=apache-beam-testing
Apr 24, 2020 1:05:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-23_18_05_50-6203930417701657601
Apr 24, 2020 1:05:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-23_18_05_50-6203930417701657601
Apr 24, 2020 1:05:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T01:05:50.520Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 24, 2020 1:05:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:05:55.203Z: Checking permissions granted to controller Service Account.
Apr 24, 2020 1:06:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:05:59.636Z: Worker configuration: n1-standard-1 in us-central1-b.
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.282Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.346Z: Expanding GroupByKey operations into optimizable parts.
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.387Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.510Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.549Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.584Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:00.633Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:01.276Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 1:06:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:01.371Z: Starting 5 workers in us-central1-b...
Apr 24, 2020 1:06:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-24T01:06:28.021Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 24, 2020 1:06:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:31.430Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 24, 2020 1:06:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:50.567Z: Workers have started successfully.
Apr 24, 2020 1:06:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:06:50.610Z: Workers have started successfully.
Apr 24, 2020 1:07:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:07:29.309Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 24, 2020 1:07:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:07:29.551Z: Cleaning up.
Apr 24, 2020 1:07:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:07:29.731Z: Stopping worker pool...
Apr 24, 2020 1:09:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:09:13.641Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 24, 2020 1:09:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-24T01:09:13.704Z: Worker pool stopped.
Apr 24, 2020 1:09:21 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-23_18_05_50-6203930417701657601 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 92975a17-e385-42e5-9781-12bb97e87b9a and timestamp: 2020-04-24T01:09:21.264000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.164
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 50.927 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 10s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/gs4br73r4kgvi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #420
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/420/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9794] Reduce state cells needed for BufferingDoFnRunner
[github] [BEAM-9701] Increments fastavro version range upper bound to 0.24.
------------------------------------------
[...truncated 284.60 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 23, 2020 6:48:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 6:48:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 23, 2020 6:48:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 6:48:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 23, 2020 6:48:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 23, 2020 6:48:23 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 23, 2020 6:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 23, 2020 6:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-FHiAPKA_CC7iy0kbYVtGCQ.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-9NsMqq2Mjm95vWy7Mc4ECw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-xahllJRZWSqArkZaJZQ10A.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-PHuHprvaH35Ag6rQB2mc6w.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-UTcx5Nae6gI-H4xdXSZhzg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7LfvFl_eIBCeGg-5N5hVvg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-kZVXj1rVUAgRdK-9xfjtoQ.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-HEFx_fuqTlJ33QrZmIObpw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-RBNqzslovRUCaDu_znp8tg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-atFLgHdT_LLypJ5Khk5qrg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Bkq8fjfoyZwIXs9q8S8Hhw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-i7rM-frGXCeN53ijCZhPCw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-cMqaqpl6HoOAVNFyK_oarQ.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-gvT1CyA0PhG8UwzfusfCow.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-uUYmc3HZshFeBxoqWUQQMw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-e6B9hNDO756bX4HOIw7hgQ.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-UgWGZx5JaUppjCQfKFmyaw.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-oIoZorTpZDAU7hhACYe9Og.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-W3rbKug7jC3C5dKwfqPNVg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-39oZg3IbKftLLvdnhjToFg.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-2FO3ZUUBtaJ-ARoGfKa4gQ.jar
Apr 23, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-M8UsrN9U2L6c659efv36dw.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-H5iOV62PD1JZn-GbPMet_A.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tVuYnFu2EnMoL83vbf6_rQ.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vTN7JOWkhPhfdos7vYWNlg.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-dqBmiMVqExh5NQNzHNY1Gg.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vTN7JOWkhPhfdos7vYWNlg.jar
Apr 23, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-v-AKR_PSdN0TLdEQahyLhQ.jar
Apr 23, 2020 6:48:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vTN7JOWkhPhfdos7vYWNlg.jar&uploadType=resumable&upload_id=AAANsUlu9VXh1szW9WTmbIFymXdexAnd_FmmNcK-xhXkEw-Lj4OJut8KZ9ugTAO6XMFQXCKqCacbw7mCDM2tA7xiqxg263EY7w.
Apr 23, 2020 6:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 23, 2020 6:48:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-vTN7JOWkhPhfdos7vYWNlg.jar
Apr 23, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 23, 2020 6:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1tnjm1wht11R3CD_DwxZKA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1tnjm1wht11R3CD_DwxZKA.pb
Apr 23, 2020 6:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-23_11_48_34-10809190354106761827?project=apache-beam-testing
Apr 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-23_11_48_34-10809190354106761827
Apr 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-23_11_48_34-10809190354106761827
Apr 23, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T18:48:34.853Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 23, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:38.017Z: Checking permissions granted to controller Service Account.
Apr 23, 2020 6:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:41.771Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.420Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.462Z: Expanding GroupByKey operations into optimizable parts.
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.494Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.563Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.599Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.627Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:42.658Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:43.009Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:48:43.080Z: Starting 5 workers in us-central1-f...
Apr 23, 2020 6:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:49:10.534Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 6:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:49:10.559Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 23, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:49:15.948Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 6:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T18:49:17.512Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 23, 2020 6:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:49:39.143Z: Workers have started successfully.
Apr 23, 2020 6:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:49:39.174Z: Workers have started successfully.
Apr 23, 2020 6:50:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:50:12.445Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 6:50:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:50:12.614Z: Cleaning up.
Apr 23, 2020 6:50:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:50:12.706Z: Stopping worker pool...
Apr 23, 2020 6:52:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:52:16.808Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 23, 2020 6:52:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T18:52:16.847Z: Worker pool stopped.
Apr 23, 2020 6:52:26 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-23_11_48_34-10809190354106761827 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c3a6add2-4aab-45de-96d5-a48514fa4d0a and timestamp: 2020-04-23T18:52:26.536000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.611
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 13.555 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 22s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/leckpfa5bqafa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #419
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/419/display/redirect>
Changes:
------------------------------------------
[...truncated 284.56 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 23, 2020 12:52:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-o8LzS6VZi0okZ_D1XPP1Gg.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-DjY27wRrfv2MSTdp0ll-Hg.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-K0_TDuotv1V28Re7efXr5w.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-SpD3SOv0qUTLY8YgSZe7cQ.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-eJYkSphUUqWs_tNxTtk6nw.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-vj6EnA6J8pQIsEHysWSXhw.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-o8xzwjwaMXv-Oyo_v0IKKA.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tzlzfR9qc5_pDbjbX--Uxw.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-x0cqDvZB4XHghfk7QCmbqQ.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-cD-za7xuYbkRwXLR0QK4VA.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-3LhnyTdpTgPeseeH6m2VLg.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-IDeS2JifLpFETMEiaZkDSg.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-2nBWpDcbo-IhwIqGpC9SWA.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-gjQKm_fAAlNlAtxErTmRRw.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-6-elGleL6FmR-GDtXKY4uQ.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-RqEW8bXc3m2DgnVMYPM3RQ.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-trsytuYoqPD2mSCZ2Fvq3w.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-IK7ZhFNVZzz6-jshm8I7Ag.jar
Apr 23, 2020 12:52:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-6Bo2DKha8IhixUhJMEd6Kg.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-FgKIdL0ffb6FO5OLkwfKYA.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-jdKsxwdQjG7HMDFPHzcjfw.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-eQ7fEmjEaioEKsS8BMF_Uw.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-FB6mZLDB700hQgWe1HjiHQ.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-MH9o2RruAtNntgvGgBhVPA.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-HDeYJHkeKLjl4nE_lYht4w.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XmZ55KxIENYJ5OPet-oHNg.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Gj58D3PcMDrM0epRrq5OaA.jar
Apr 23, 2020 12:52:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XmZ55KxIENYJ5OPet-oHNg.jar
Apr 23, 2020 12:52:34 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XmZ55KxIENYJ5OPet-oHNg.jar&uploadType=resumable&upload_id=AAANsUktyCIFQ_dlslLjax4FhLvI5jsNR4PJE7HlWDXlrILNKbMIEmhfFOJmokDm4C8aZCwiV6xQcLvIo6FNzaVAb_gv93geKQ.
Apr 23, 2020 12:52:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 23, 2020 12:52:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-XmZ55KxIENYJ5OPet-oHNg.jar
Apr 23, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 23, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash Pg_YdrbHKLJoWrMAZBPOcQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Pg_YdrbHKLJoWrMAZBPOcQ.pb
Apr 23, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 23, 2020 12:52:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-23_05_52_43-721726139909694547?project=apache-beam-testing
Apr 23, 2020 12:52:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-23_05_52_43-721726139909694547
Apr 23, 2020 12:52:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-23_05_52_43-721726139909694547
Apr 23, 2020 12:52:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T12:52:43.109Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 23, 2020 12:52:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:46.586Z: Checking permissions granted to controller Service Account.
Apr 23, 2020 12:52:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:50.992Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:51.744Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:51.791Z: Expanding GroupByKey operations into optimizable parts.
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:51.831Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:51.962Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:51.999Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:52.054Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:52.111Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:52.645Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:52:52.726Z: Starting 5 workers in us-central1-f...
Apr 23, 2020 12:53:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T12:53:04.364Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 23, 2020 12:53:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:53:27.677Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 12:53:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:53:50.766Z: Workers have started successfully.
Apr 23, 2020 12:53:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:53:50.807Z: Workers have started successfully.
Apr 23, 2020 12:54:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:54:22.020Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 12:54:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:54:22.294Z: Cleaning up.
Apr 23, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:54:22.434Z: Stopping worker pool...
Apr 23, 2020 12:55:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:55:58.865Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 23, 2020 12:55:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T12:55:58.914Z: Worker pool stopped.
Apr 23, 2020 12:56:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-23_05_52_43-721726139909694547 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 72114a6d-9298-4ead-93f3-9997dcd2d8db and timestamp: 2020-04-23T12:56:06.183000000Z:
Metric: Value:
read_time 13.893
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 44.559 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 33s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/ocdfkb2guv4fa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #418
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/418/display/redirect>
Changes:
------------------------------------------
[...truncated 284.52 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 23, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 23, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 6:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 23, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 23, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 23, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 23, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-McAsk5KixVEaLGBjVDeboQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-6kgV3O_--d49MhrCOX-7pg.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-CVVMvXGzoWTN0OuK2q1t0A.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Go1KPIP2syKHZhZ6Pnzvnw.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-LsyNGM8uyYOKGBAI7W4jgg.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-EQfeYDViM5DhXu4BD3jdnw.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-e55OzckZjiRlJWBmAUFhlQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Gjko02LR4Uis26zBZI4OCQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-xWU-rmGlAJ84Wvtua-YDMQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-elYXUph7Tlt66kbbt3YW4w.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-_x98ZbufenAviHBw8Bcbbg.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-psKyqNVbEXk098M3E7VWmw.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-l5q6SgriB_Si52RoLYx2Xg.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-LCiAbIRysIe5fh7LAM2CgA.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-I0_T4JnsLm20QqNwKKGO4g.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-4tlv6UlsMlDCYzL3RFU9Ew.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kXkG3M4T9YMXgiiZCj8Cyg.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-KzjkMimzLMUEvOYVervX8A.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-S0uJLEsD7HvKaIRz27b0XA.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-8HUInhu5fKhs5rdGpWIbfQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-MjHUlQwsGfbDeIaqUIjqGw.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-ZZOJG0ut0sh8JHoLnKovSA.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-UXzXT57kN5lqhMn8BptjWQ.jar
Apr 23, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-7W5yq1lJ4JU9JvSSFaPHCg.jar
Apr 23, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lWBQKGqLwSszepdUukmI4w.jar
Apr 23, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-U0AehasGSyrAv_ix3-bh0g.jar
Apr 23, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-66F-ysbpuGBQdg0tbHSwbg.jar
Apr 23, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lWBQKGqLwSszepdUukmI4w.jar
Apr 23, 2020 6:45:36 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lWBQKGqLwSszepdUukmI4w.jar&uploadType=resumable&upload_id=AAANsUk9cI2Laj49cbFOTEisN6Bz4smQaRzk4ZTaHN0Wig44m3CZgVwNhl21DkahHEoSQSNzfYvyFvjac27lklKYpKI.
Apr 23, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 23, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-lWBQKGqLwSszepdUukmI4w.jar
Apr 23, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 23, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash yHZL9RgCOZ7hufePfn5gJA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yHZL9RgCOZ7hufePfn5gJA.pb
Apr 23, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 23, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-22_23_45_41-2830485910220148624?project=apache-beam-testing
Apr 23, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-22_23_45_41-2830485910220148624
Apr 23, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-22_23_45_41-2830485910220148624
Apr 23, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T06:45:41.817Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 23, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:44.966Z: Checking permissions granted to controller Service Account.
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:49.641Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.241Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.273Z: Expanding GroupByKey operations into optimizable parts.
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.301Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.379Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.416Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.455Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.482Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.831Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:45:50.913Z: Starting 5 workers in us-central1-f...
Apr 23, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T06:45:58.896Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 23, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:46:19.242Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:46:37.816Z: Workers have started successfully.
Apr 23, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:46:37.853Z: Workers have started successfully.
Apr 23, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:47:14.573Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:47:14.786Z: Cleaning up.
Apr 23, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:47:14.881Z: Stopping worker pool...
Apr 23, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:48:37.987Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 23, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T06:48:38.024Z: Worker pool stopped.
Apr 23, 2020 6:48:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-22_23_45_41-2830485910220148624 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 6abf8516-6cb7-45fa-9f77-49d16831a74f and timestamp: 2020-04-23T06:48:43.727000000Z:
Metric: Value:
read_time 18.826
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 19.804 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/tfhjdomrcfimw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #417
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/417/display/redirect?page=changes>
Changes:
[github] [BEAM-9577] Use new artifact API to stage go artifacts. (#11490)
[github] Force output names to be unicode-compatible on Python 2. (#11485)
[github] Make license_script.sh tolerate directory already existing (#11501)
[github] [BEAM-9778] remove temp directory & add docker-pull-licenses to release
[github] [BEAM-8603] Add Python SqlTransform (#10055)
------------------------------------------
[...truncated 284.81 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 23, 2020 12:54:02 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-YmOYkHtXVH_2y8TnmwVQxA.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-nfsKfCaHksOqpd83d5NnrQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-tarGldUxeW84jEJpjtt_vw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-EXvfuXQH1uDSQzo0jY1XUw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-2YUhk255tedSUOn993r8Iw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Jwebnd2IfsfuyoajasNGJA.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-wnMT8JbUWvfRMya-e2hFXg.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-LRvzbxEig_TjBFQro7G3rA.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-hpseVBn8aTNDX6dzpd4rwg.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-FI2_0DT6lsNfn0T5OU_CYg.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-XcAQtfv7A93aSUuoaEMLvw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-xm4cE_DFzmcRcOqkgZBxRw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-v444Sk6vTObXO5wKpEvtGw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ndN8TSgaQBTbJSnbZmz9MQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-r7jX0Zq7a7WE5OPYOfYC1g.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-DFoessLxog4Gpr1l_oWpHw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-XG6HrX4fqfGokINoobSzaw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_9moHfzW1dDS83GW27LdbQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-62lcW5qg5k5cFAGrNUIEyw.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-lodBY2wVJB3xpAYip91BeQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-100CfpYQEKqPJHfHXsRwcQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-qnIE-M0fZ5OOgVWbVh0xBQ.jar
Apr 23, 2020 12:54:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-C3huNtlQRONo9lidgbU9CQ.jar
Apr 23, 2020 12:54:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-UDEluCH9dPq5ssd6dD2cvw.jar
Apr 23, 2020 12:54:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-3CcOslFNbuE7d18do8_9Yg.jar
Apr 23, 2020 12:54:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HglrNNKngebc8T5ume5pIA.jar
Apr 23, 2020 12:54:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-KDQQUxWVsDMn8afnKLpGig.jar
Apr 23, 2020 12:54:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HglrNNKngebc8T5ume5pIA.jar
Apr 23, 2020 12:54:06 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HglrNNKngebc8T5ume5pIA.jar&uploadType=resumable&upload_id=AAANsUkNS74fx-tnOZHqdbZ7U7A-78kUeqr0tAu1Dm5AzlY5wvUmWVR1crCPsbMb4l52rWgcOIAHi-wf3n02i4oDFMQ.
Apr 23, 2020 12:54:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 23, 2020 12:54:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HglrNNKngebc8T5ume5pIA.jar
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 23, 2020 12:54:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash cM455Tbt9Ko8bmBEyyXXOA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cM455Tbt9Ko8bmBEyyXXOA.pb
Apr 23, 2020 12:54:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 23, 2020 12:54:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-22_17_54_12-16039288066080705652?project=apache-beam-testing
Apr 23, 2020 12:54:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-22_17_54_12-16039288066080705652
Apr 23, 2020 12:54:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-22_17_54_12-16039288066080705652
Apr 23, 2020 12:54:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T00:54:12.565Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 23, 2020 12:54:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:17.661Z: Checking permissions granted to controller Service Account.
Apr 23, 2020 12:54:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.121Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.830Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.861Z: Expanding GroupByKey operations into optimizable parts.
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.885Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.967Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:21.994Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:22.019Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:22.047Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:22.338Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 12:54:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:22.399Z: Starting 5 workers in us-central1-f...
Apr 23, 2020 12:54:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:50.237Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 12:54:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:50.299Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 23, 2020 12:54:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-23T00:54:52.583Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 23, 2020 12:54:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:54:55.787Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 23, 2020 12:55:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:55:08.618Z: Workers have started successfully.
Apr 23, 2020 12:55:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:55:08.642Z: Workers have started successfully.
Apr 23, 2020 12:55:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:55:41.111Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 23, 2020 12:55:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:55:41.301Z: Cleaning up.
Apr 23, 2020 12:55:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:55:41.376Z: Stopping worker pool...
Apr 23, 2020 12:57:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:57:35.100Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 23, 2020 12:57:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-23T00:57:35.143Z: Worker pool stopped.
Apr 23, 2020 12:57:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-22_17_54_12-16039288066080705652 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 89eedb09-519a-4adc-866e-3f1044c59daf and timestamp: 2020-04-23T00:57:42.623000000Z:
Metric: Value:
read_time 16.14
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 48.833 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/n7teezplx5ihe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #416
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/416/display/redirect?page=changes>
Changes:
[markliu] [BEAM-8831] Do not copy generated files for Python source copy
[github] [BEAM-6597, BEAM-6467] Add support for reporting monitoring infos to the
[yoshiki.obata] [BEAM-9755] changed MAKE_ADD stack pop order according to Python version
[github] Merge pull request #11489 [BEAM-9577] Ensure required directories exist
[github] [BEAM-9642] Enabling SDF exec runners and fixing small bug. (#11474)
[github] [BEAM-9717] update future version to 0.18.2 for Python 3.8 (#11494)
------------------------------------------
[...truncated 283.10 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 22, 2020 7:43:44 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 7:43:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 22, 2020 7:43:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 7:43:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 22, 2020 7:43:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 22, 2020 7:43:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 22, 2020 7:43:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 22, 2020 7:43:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-ZXmuY8hG4ZhzvWd-73L_-Q.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-c3QOsy-74n545XFNy-fxzQ.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-KBQHFDziqty6iYIs6APrcA.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ZpbAWSlm5qSRxQxJ8hmScw.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-wAnbGwOrcMxQtOCO4fJ7OA.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-eGkNVzMPzZXAfvuLl3t4Cg.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ucRUu9RbHIsOv-TtfCBVHA.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-EX1AJf48J5b99yFg_qxB6g.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-q34QmisaD4SEoz314irP_Q.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-CYbV2ue7dafvBPGDSfJsDw.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-nMzLt161YsceTYRZ4CKcFg.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-eLNbKlombPivxmQb5AeBUw.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-EryGcw0PMtkw6iCvmZ0_dw.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-S6MdIstKGYjfehWda2vRbQ.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-k1apN_ufPhkad-Ywz35yyw.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-5bkhUpBgTCiUqgYGnOkqlg.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-C7XL4x4HpjyO2XeMXrZf1w.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-FgshACQ8Jek8U1inAKzJ0Q.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-6OUlJJgqJUHgeSNEqZ4zyg.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-rpss_N_GevLJ13ypu_HfxQ.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ZypSpU8XWeQ3gV9ipYRmjA.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-qqXIGSPVHCwaIEsQX139Kg.jar
Apr 22, 2020 7:43:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-FDbpWME93hW7dWI_wgNrLA.jar
Apr 22, 2020 7:43:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bxiEdcMopOM0bISqqLQ5tg.jar
Apr 22, 2020 7:43:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-1fZSKHL5z_gKAeFDJndeTw.jar
Apr 22, 2020 7:43:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-RkpxXwzweSHck30lO6QJ3w.jar
Apr 22, 2020 7:43:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-f6HFZCyzJm2K6z4TOIjUyQ.jar
Apr 22, 2020 7:43:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bxiEdcMopOM0bISqqLQ5tg.jar
Apr 22, 2020 7:43:48 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bxiEdcMopOM0bISqqLQ5tg.jar&uploadType=resumable&upload_id=AAANsUksVghlsHkjnkm55Kr57HNbQlGXpFKzxsyToVqvYjkWuToQaNcfgOIukT4y8uFfjU3rixCLgcIlBZCKFUpVex8RGFZH0g.
Apr 22, 2020 7:43:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 22, 2020 7:43:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bxiEdcMopOM0bISqqLQ5tg.jar
Apr 22, 2020 7:43:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 22, 2020 7:43:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash toRP_G8KZYqWrNgWFp8IBg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-toRP_G8KZYqWrNgWFp8IBg.pb
Apr 22, 2020 7:43:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 22, 2020 7:43:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-22_12_43_57-7802715387781168082?project=apache-beam-testing
Apr 22, 2020 7:43:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-22_12_43_57-7802715387781168082
Apr 22, 2020 7:43:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-22_12_43_57-7802715387781168082
Apr 22, 2020 7:43:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T19:43:57.929Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 22, 2020 7:44:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:01.617Z: Checking permissions granted to controller Service Account.
Apr 22, 2020 7:44:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:04.794Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.446Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.475Z: Expanding GroupByKey operations into optimizable parts.
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.586Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.645Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.664Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.689Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:05.717Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:06.033Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 7:44:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:06.091Z: Starting 5 workers in us-central1-f...
Apr 22, 2020 7:44:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T19:44:32.342Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 22, 2020 7:44:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:42.398Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 7:44:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:42.433Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 22, 2020 7:44:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:44:47.826Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 7:45:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:45:01.910Z: Workers have started successfully.
Apr 22, 2020 7:45:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:45:01.943Z: Workers have started successfully.
Apr 22, 2020 7:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:45:35.988Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 7:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:45:36.213Z: Cleaning up.
Apr 22, 2020 7:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:45:36.305Z: Stopping worker pool...
Apr 22, 2020 7:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:47:17.145Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 22, 2020 7:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T19:47:17.182Z: Worker pool stopped.
Apr 22, 2020 7:47:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-22_12_43_57-7802715387781168082 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 45ef5bf0-3d59-4d55-a0b3-454307d04585 and timestamp: 2020-04-22T19:47:23.368000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.271
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 48.73 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ymj4o3a2i3dii
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #415
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/415/display/redirect>
Changes:
------------------------------------------
[...truncated 284.08 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 22, 2020 12:52:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 22, 2020 12:52:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 22, 2020 12:52:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-52XyK6GtkzxfA4dzkVsfUA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-fzbUcd5g7_eChqgtpPegrA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-UV6KOOWAA5TsA3gmFAORaw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Jp2bcMcu9usZ1Rr51aKXfw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-UDjD3GGW0R4WSnPieEjI8g.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-DwujK6idg6iV0i1SpUIriA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT--pHDRSIe8M1siF3-cK3apA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-ZcekEs8hA71l-gzBZevssQ.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-J0LXe-77YF_ywvR-76lQbw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-0CIaOpAa0RwYVQHZE6HVxg.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-RRZjLbHVTGk2yjFmQacE4g.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-IeCf0VDzUmo7wozuHAXQ6w.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-oPWl6J_MJGsf8-iSrvtw7A.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-cqcRSPruHynzyIAWN6s1kg.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-RppPUWPNc835OfcK7HSs9w.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-GD6bqyxGYm9pjJUvL5__FQ.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-spysI3oQ29gely69VEFgEw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_oehf9LLd9dNzd4YudCZhA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-xpMu0sTEW7CWbvD3TvvHyg.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-nWlc9VN6-SdppG8_SLuX4Q.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-jI7YCr5DviIAptIYje0VwA.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-m3a_JrUa8wL0_2OhSjLjTw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-GcJL3enn0KlKGThAiVaEJw.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-9NPOiPNycXwA8Py96fxmMg.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-hosxyQYeyd0Q73z2me2YYQ.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-mEJUaIwby3V4SvJ26rV0Gg.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-MV-8ioPq44GPY9UmynErmQ.jar
Apr 22, 2020 12:52:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-MV-8ioPq44GPY9UmynErmQ.jar
Apr 22, 2020 12:52:38 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-MV-8ioPq44GPY9UmynErmQ.jar&uploadType=resumable&upload_id=AAANsUkR4L-MdwFLEe6K2axzGa-cH_JMapSP8VZ47Q_qiUYgsXsSLnfOIYT_4t6kUvGT7dcOZ8Ygvq9RRfio20r56Og.
Apr 22, 2020 12:52:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 22, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-MV-8ioPq44GPY9UmynErmQ.jar
Apr 22, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash SmxJzE-DXJajL-rsDBd_Gw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SmxJzE-DXJajL-rsDBd_Gw.pb
Apr 22, 2020 12:52:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 22, 2020 12:52:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-22_05_52_43-14103965762991753285?project=apache-beam-testing
Apr 22, 2020 12:52:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-22_05_52_43-14103965762991753285
Apr 22, 2020 12:52:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-22_05_52_43-14103965762991753285
Apr 22, 2020 12:52:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T12:52:44.005Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 22, 2020 12:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:47.122Z: Checking permissions granted to controller Service Account.
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:51.416Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.072Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.111Z: Expanding GroupByKey operations into optimizable parts.
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.146Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.222Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.252Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.286Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.308Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.663Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:52:52.742Z: Starting 5 workers in us-central1-f...
Apr 22, 2020 12:53:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T12:53:06.166Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 22, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:53:19.172Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:53:19.210Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 22, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:53:24.758Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 12:53:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:53:39.842Z: Workers have started successfully.
Apr 22, 2020 12:53:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:53:39.877Z: Workers have started successfully.
Apr 22, 2020 12:54:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:54:17.084Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 12:54:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:54:17.324Z: Cleaning up.
Apr 22, 2020 12:54:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:54:17.410Z: Stopping worker pool...
Apr 22, 2020 12:56:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:56:03.771Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 22, 2020 12:56:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T12:56:03.822Z: Worker pool stopped.
Apr 22, 2020 12:56:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-22_05_52_43-14103965762991753285 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 07735997-f5ee-4bd3-9b1c-787f8902200a and timestamp: 2020-04-22T12:56:09.672000000Z:
Metric: Value:
read_time 19.563
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 44.031 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 56s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ahgfxzw2jraqc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #414
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/414/display/redirect?page=changes>
Changes:
[github] Log job id similar to Java and Go
------------------------------------------
[...truncated 283.13 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 22, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 22, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-n_adUvuSNRZgkUo1272-XA.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-j4yUieU8N4BuEL_lgYR2fw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-I-StzWL-N1cLqhEGd7zq4g.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-QoFFuSjK5f6vl-SO36B85w.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-KmAldwZvUTMFkb6iOIOS1A.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_GqS77ZakPKGXSt3XmrU8g.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-xSLbxVkmCVZtW2BDy_ZDwg.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-bsrrj4h-fZKDDGescrNBnA.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-vnUXPP8FmUElfpyxMEpHaw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-FQL5XgdI0UgDasrJpjHpHw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-r6p20Od-Kr_nvhRTclAwCA.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-j9yfM17Jo7XRlf8MW92f7Q.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT--ZshtjlHFQ5hZSXSNr8dww.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-qczg_v0FJaCIMQi49aTWew.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-t4jjS09BiJvmG-KXnIw9yQ.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-N9mxDKpXzZXm6gDVBC9hWg.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-XCqKoFMDH03wgUVOj39hHw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-RIfRdrZv3gsN2UQ4U73FdQ.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-dv1JO0ZlHQIpJ9r_ljuIPA.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-k2eXPAQw-wFtZktWIgQMlA.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-d6x_xkK_bLHxR8Hcsqfabg.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-NpnkISpgKoqQZ0nEenE0Fw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-diVorg37dE8W8yd1VRt48A.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-diVorg37dE8W8yd1VRt48A.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-2ROo_6JWza4euE3Srk-Wlw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-QVlHBe2qGfveXcd-izpggw.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-8UDDzlkkCArLkF5pt_wA2Q.jar
Apr 22, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-qBA0deMMqExqdTdjhwsrjg.jar
Apr 22, 2020 6:45:40 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-diVorg37dE8W8yd1VRt48A.jar&uploadType=resumable&upload_id=AAANsUnIzcf_kY9i6IEfCbQuFcrlm-KtI36TknsXuVZFodQN8-eN4_k0NsYxe0XoWVWyRajoM6MMh8qtnX-pcGAYeFxIr7AGfw.
Apr 22, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 22, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-diVorg37dE8W8yd1VRt48A.jar
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 22, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash xBRVeqallaDI4Id1B5FzTw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xBRVeqallaDI4Id1B5FzTw.pb
Apr 22, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-21_23_45_48-17662694227698529284?project=apache-beam-testing
Apr 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-21_23_45_48-17662694227698529284
Apr 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-21_23_45_48-17662694227698529284
Apr 22, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T06:45:48.387Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 22, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:51.487Z: Checking permissions granted to controller Service Account.
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:54.785Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.452Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.487Z: Expanding GroupByKey operations into optimizable parts.
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.513Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.608Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 22, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.652Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 22, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.684Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 22, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:55.719Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 22, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:56.157Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:45:56.239Z: Starting 5 workers in us-central1-a...
Apr 22, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:46:24.490Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:46:24.528Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 22, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T06:46:26.556Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 22, 2020 6:46:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:46:29.902Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:46:52.723Z: Workers have started successfully.
Apr 22, 2020 6:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:46:52.761Z: Workers have started successfully.
Apr 22, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:47:30.381Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:47:30.580Z: Cleaning up.
Apr 22, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:47:30.667Z: Stopping worker pool...
Apr 22, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:49:11.807Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 22, 2020 6:49:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T06:49:11.849Z: Worker pool stopped.
Apr 22, 2020 6:49:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-21_23_45_48-17662694227698529284 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 34d9ad22-b0af-4282-8038-d229b33c2941 and timestamp: 2020-04-22T06:49:19.198000000Z:
Metric: Value:
read_time 16.003
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 51.567 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 2s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/sm2ucdx3dnczg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #413
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/413/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-9608] BundleManagers use BundleContextManager for configuration
[rohde.samuel] Add a timeout to the TestStream GRPC and fix the Streaming cache timeout
[pabloem] [BEAM-9639] Saving side inputs after producer execution, not before
[pabloem] [BEAM-9639] Separate Stage and Bundle execution. Improve typing
[boyuanz] Fix SplittableParDoNaiveBounded DoFnInvoker.
[shoaib.zafar] update release 2.20.0 blog post and added contributors name
[pabloem] [BEAM-9639][BEAM-9608] Addressing review comments.
[github] Revert "[BEAM-9014] CachingShuffleBatchReader use bytes to limit cache
[github] [BEAM-9789] Fix lock error. Add test. (#11468)
[github] [BEAM-7981] Correct ParDo(callable) test (#11446)
------------------------------------------
[...truncated 282.75 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 22, 2020 12:59:56 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 12:59:56 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 22, 2020 12:59:56 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 22, 2020 12:59:56 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 22, 2020 12:59:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 22, 2020 12:59:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 22, 2020 12:59:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 22, 2020 12:59:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-RndNiaiaNlmEAsN7n6qfUA.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-h2CYQ4aHpmBDrDTLiZDW0w.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-IeoCwXskIprcPPBeQQnvnw.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-dviQuJUeUnkkmGek43rasw.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-8tZg3LwkaPHWx8uk7HxRug.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-MMdqDiSpPH7ZCRkFnxmQdA.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-lxaR5FZoFES9LZOEyvTYzw.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-heXC9tV5htXLlw4cZkn5mQ.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-il9V9E-9chB46f5HaizHkA.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-4qGlF8r1Ku86Yn9oiJsoVQ.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-m9WNDnUItXhipVz67jkVYw.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-eiNjTqa66skfOwF3sc4zaQ.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Uzm7mGdvtoP3ayh2_4DQ8g.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-mr4nbW1SzjTR9DZguGUM4A.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Qqo4LBP9Dd9ai_apItKvUQ.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-SklMzXn2yfJ2DJRxKTvMDg.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-mvninosUQv1A4taV2o9BIA.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tLzxc3RLOUGGsNCBIfKFog.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-fSteONu7s5ZfDRHMYO5u2g.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-KunPUVOu5NLipf3BGVJxMg.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-yGs9yJQTvEIf2S-7D1fbFQ.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m3TzSIPER7p0dO8DzE8iog.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-BfaJmITGSeZxJdriHZnNKg.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-ob11qis0WsGgHsh58HVy-A.jar
Apr 22, 2020 12:59:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-2EBteD0dqEo1jx4aK7rImA.jar
Apr 22, 2020 12:59:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-SVTP1huD7_u7rWag0z38BA.jar
Apr 22, 2020 12:59:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-bkSfAS5A92FUF-JpCRXnoQ.jar
Apr 22, 2020 12:59:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m3TzSIPER7p0dO8DzE8iog.jar
Apr 22, 2020 1:00:00 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m3TzSIPER7p0dO8DzE8iog.jar&uploadType=resumable&upload_id=AAANsUmY-1XmDQIw0eWXVwwHVg7J9BA7hWKEdyIFqVR5msTW5-xeACyYDvfvhmoX1bI3ZvsJUkD-S-i6ZJpiB9A7GQM9DDdWLw.
Apr 22, 2020 1:00:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 22, 2020 1:00:07 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m3TzSIPER7p0dO8DzE8iog.jar
Apr 22, 2020 1:00:08 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 22, 2020 1:00:08 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash t80n90Y6718XrYY73FDtkA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-t80n90Y6718XrYY73FDtkA.pb
Apr 22, 2020 1:00:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 22, 2020 1:00:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-21_18_00_09-9622795192956306543?project=apache-beam-testing
Apr 22, 2020 1:00:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-21_18_00_09-9622795192956306543
Apr 22, 2020 1:00:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-21_18_00_09-9622795192956306543
Apr 22, 2020 1:00:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T01:00:09.839Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 22, 2020 1:00:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:14.098Z: Checking permissions granted to controller Service Account.
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:18.690Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.452Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.502Z: Expanding GroupByKey operations into optimizable parts.
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.553Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.687Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.715Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.756Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 22, 2020 1:00:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:19.797Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 22, 2020 1:00:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:20.358Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 1:00:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:20.458Z: Starting 5 workers in us-central1-f...
Apr 22, 2020 1:00:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-22T01:00:46.607Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 22, 2020 1:00:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:52.319Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 1:00:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:52.361Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 22, 2020 1:01:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:00:57.846Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 22, 2020 1:01:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:01:08.144Z: Workers have started successfully.
Apr 22, 2020 1:01:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:01:08.187Z: Workers have started successfully.
Apr 22, 2020 1:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:01:46.216Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 22, 2020 1:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:01:46.441Z: Cleaning up.
Apr 22, 2020 1:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:01:46.519Z: Stopping worker pool...
Apr 22, 2020 1:03:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:03:15.295Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 22, 2020 1:03:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-22T01:03:15.353Z: Worker pool stopped.
Apr 22, 2020 1:03:21 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-21_18_00_09-9622795192956306543 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 59248b7e-3c4f-4e7d-8f72-dc393dde0742 and timestamp: 2020-04-22T01:03:21.792000000Z:
Metric: Value:
read_time 20.293
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 33.062 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 36s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/kznlmz6ozgs7s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #412
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/412/display/redirect?page=changes>
Changes:
[ankurgoenka] [BEAM-9791] Add precommit for dataflow runner v2
[apilloud] [BEAM-9662][BEAM-9663] Reject Null Limit and Offset
[github] Revert "[BEAM-8131] Provide Kubernetes setup for Prometheus (#9482)"
[github] Merge pull request #11466 from [BEAM-9787] Clear error message on UW +
[github] [BEAM-9764] multi threading & optional pulling (#11428)
------------------------------------------
[...truncated 282.71 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 21, 2020 7:07:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-eU68hBp0KmpHu7Q0W8ZNkQ.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-_SjToYfzJX_vD-QbmiQI3g.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-XNvIzvbTItjpb8n8j67kvg.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Dn5m0OdLsfLrBE7Bq3ajJA.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-wxoW7rTA-FkG0Qp_xDfPJw.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-zX1TwLYpU24LKri28wkDgw.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-GpNPmGMK3SZSPNnGM6VP_A.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-3MYWmGALbIGjGDaMVA1_Qg.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-VbhZ_mbigJMrS4Z2k3KftA.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT--I6yjMn3Biu4oyXvsAN5HA.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-kEcOafTk4DgpY71KzQjVkw.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-FzKKFPQMMwTXOEN8Ahk0bA.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-D-NkjgdHvLFDQGOVOoY16Q.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ejXiIYLrCA3YGNtzLN81gg.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-WaWTXG6YddIrH8zlNOGUIg.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-MxxLOc94MUlEYjXyhEgWXw.jar
Apr 21, 2020 7:07:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-wMK8_LToYHTg408ZBns7xw.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-er6SByhpd2W0WGMecs_Stw.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-6nnKYPaPUkagYtcE2nnp4w.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-SulgNbk-xNdjmkl-rLUH9A.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-dY6vb9SvTPMZ4BZcvVa-pw.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-P5pAYmDYyfNez4N3t4Wg1w.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-SAt9tDUZiyNthQSL53bAgA.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-q5X7C16XfniHkVo3-1JFOg.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-1h4aUzH1_IZsxp9K3oZruA.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-qedQSIO0O6PzHcCPEpslww.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-q5X7C16XfniHkVo3-1JFOg.jar
Apr 21, 2020 7:07:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-XxkKHZhFqCA9fmXaOW-aYg.jar
Apr 21, 2020 7:07:59 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-q5X7C16XfniHkVo3-1JFOg.jar&uploadType=resumable&upload_id=AAANsUkjEifCnmWJvMpNZ_jSoz8OnQDWNVEW0yMXy7RWeisOF5cQoXgO9mRBQGbbCEDSp-rwmRL9By9NdXA8BDJJHKI.
Apr 21, 2020 7:07:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 21, 2020 7:08:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-q5X7C16XfniHkVo3-1JFOg.jar
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 21, 2020 7:08:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 21, 2020 7:08:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash vugAUWwD3IW03OhrSCi9rg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vugAUWwD3IW03OhrSCi9rg.pb
Apr 21, 2020 7:08:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 21, 2020 7:08:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-21_12_08_04-13147726653383608999?project=apache-beam-testing
Apr 21, 2020 7:08:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-21_12_08_04-13147726653383608999
Apr 21, 2020 7:08:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-21_12_08_04-13147726653383608999
Apr 21, 2020 7:08:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T19:08:04.597Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 21, 2020 7:08:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:08.030Z: Checking permissions granted to controller Service Account.
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:12.597Z: Worker configuration: n1-standard-1 in us-central1-b.
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.260Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.303Z: Expanding GroupByKey operations into optimizable parts.
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.342Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.427Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.470Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.509Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.547Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:13.994Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 7:08:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:14.081Z: Starting 5 workers in us-central1-b...
Apr 21, 2020 7:08:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T19:08:28.515Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 21, 2020 7:08:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:37.659Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 7:08:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:59.128Z: Workers have started successfully.
Apr 21, 2020 7:08:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:08:59.167Z: Workers have started successfully.
Apr 21, 2020 7:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:09:36.350Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 7:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:09:36.537Z: Cleaning up.
Apr 21, 2020 7:09:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:09:36.624Z: Stopping worker pool...
Apr 21, 2020 7:12:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:12:06.622Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 21, 2020 7:12:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T19:12:06.675Z: Worker pool stopped.
Apr 21, 2020 7:12:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-21_12_08_04-13147726653383608999 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 63262f2e-750f-4a2c-8ce9-e33b6ca11455 and timestamp: 2020-04-21T19:12:12.231000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.049
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 4 mins 25.117 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 26s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/fccgnofgabfqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #411
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/411/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-9285] Add Postcommit ValidatesRunner CI Job for Flink on Java 11
------------------------------------------
[...truncated 291.92 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 21, 2020 1:00:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 1:00:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 21, 2020 1:00:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 1:00:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 21, 2020 1:00:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 21, 2020 1:00:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 21, 2020 1:00:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 21, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-VhH8S9dSTbpwkB6UXdr4tw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-C7FuzAI7m7dc6Y3xlPXIig.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-fN2MEfZPAokblQqiXmajLw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-9l6dkLSHcLnyjPauRHPSFA.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-AcMMT_sRfpqPcyg2YMm0uw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-NPnxZ91792NUEMMQ7bLGBA.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-bH8eb3rHEllSBblF9mw-rw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-xzw3Is0FyZHX1OzO8cjL9Q.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-n8FpgVoufSVpwNA51PND-w.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-6-PHaHTn_bOFlCaTjATRbw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-BqWUZZdfKWN52KWm6jITUQ.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-XVKGamggPTslDQ_FJv3Fyg.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Oce2Jhae_weABBiknn8oSA.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-GxuW2GRHSppgiaV-XXNztw.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-9Gl7IckRYOzOsE-M8RYuJg.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-RF5OHV5TUBklJmCTP-egrg.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ZaN0ekF_ugG8WlbTP5g18w.jar
Apr 21, 2020 1:00:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-sPs_xOv-CyNv29ZQq4ROew.jar
Apr 21, 2020 1:00:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-A0It4lwYLOVZKp2DANcJTg.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-b4NNRH3WErk4MYPRdvzEiQ.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-grICnFWfFw73BtK8Jt0utg.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-bQDKa4tZIX5Xf1KsYSqFDw.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-_ZnLSIwI46XA7qa4VlPoeQ.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-BqWUZZdfKWN52KWm6jITUQ.jar
Apr 21, 2020 1:00:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-gFb1sAG9RvmkD6w5UOKrxA.jar
Apr 21, 2020 1:00:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-G7W-9S9Onyoyywf_8a8_VQ.jar
Apr 21, 2020 1:00:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-KFRpcvAN8mQDbDTyik_vFA.jar
Apr 21, 2020 1:00:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-3yL-ddP2tqbNicjj3SQjtg.jar
Apr 21, 2020 1:00:12 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-BqWUZZdfKWN52KWm6jITUQ.jar&uploadType=resumable&upload_id=AAANsUnNOdjzJX1K3BzqZEpnzMed5A2hAORdixPeEysy97p1bGTB6KfhYk-84OKcG3aqMSwJBNMF8dqW-NYEd5E3irttXX8w1g.
Apr 21, 2020 1:00:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 21, 2020 1:00:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-BqWUZZdfKWN52KWm6jITUQ.jar
Apr 21, 2020 1:00:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 11 seconds
Apr 21, 2020 1:00:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 21, 2020 1:00:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 21, 2020 1:00:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 21, 2020 1:00:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 21, 2020 1:00:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 21, 2020 1:00:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash x3cQRUrgo1pQ091fGwPNhA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-x3cQRUrgo1pQ091fGwPNhA.pb
Apr 21, 2020 1:00:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 21, 2020 1:00:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-21_06_00_19-3159029043915494442?project=apache-beam-testing
Apr 21, 2020 1:00:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-21_06_00_19-3159029043915494442
Apr 21, 2020 1:00:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-21_06_00_19-3159029043915494442
Apr 21, 2020 1:00:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T13:00:19.631Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 21, 2020 1:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:23.458Z: Checking permissions granted to controller Service Account.
Apr 21, 2020 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:27.442Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 21, 2020 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.169Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.206Z: Expanding GroupByKey operations into optimizable parts.
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.243Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.314Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.355Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.391Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.429Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.810Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 1:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:28.884Z: Starting 5 workers in us-central1-f...
Apr 21, 2020 1:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T13:00:35.233Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 21, 2020 1:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:54.159Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 1:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:54.197Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 21, 2020 1:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:00:59.634Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 1:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:01:12.679Z: Workers have started successfully.
Apr 21, 2020 1:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:01:12.703Z: Workers have started successfully.
Apr 21, 2020 1:01:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:01:43.869Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 1:01:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:01:44.140Z: Cleaning up.
Apr 21, 2020 1:01:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:01:44.254Z: Stopping worker pool...
Apr 21, 2020 1:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:03:41.160Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 21, 2020 1:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T13:03:41.206Z: Worker pool stopped.
Apr 21, 2020 1:03:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-21_06_00_19-3159029043915494442 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8833d8c6-a616-4111-9aca-807ad0ff9837 and timestamp: 2020-04-21T13:03:47.147000000Z:
Metric: Value:
read_time 13.668
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.15 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.134 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 4 mins 15.552 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 49s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/mij2jjngdux3s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #410
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/410/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-6960] More sophisticated job server starting and stopping.
[robertwb] Abort pipeline if docker containers die unexpectedly.
[robertwb] [BEAM-6960] Go ULR validates runner tests.
[robertwb] lint
------------------------------------------
[...truncated 282.85 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 21, 2020 6:47:10 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 21, 2020 6:47:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 21, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-_16ByOjUt_rnhtqDHHb6gg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-gFB2ozWCBiKtGtdjceerAQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-uOLeCmU7c3DhtqH5EPGvsA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-B36xPU5jzxpyJz9p6L8_qA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT--X1g0AdOn3MK1eZF-Cm_Eg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-LnVU7ROZ0dJH_rr2biTxFA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-9Kx0xw3FIawxDujpkm63xg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-96_sXzvr3Vdbstf80HJqLw.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-WY3XTgSV7_e0-J9jrfgXJw.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-WQY6ut4YZqCSasu2gnGTWA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-2U1jKO5O_Cgbdm2T2DU_nA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-jHbmuNuY-lIf1uhisC5PfQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-5pY6rZdEcKdEIlE6-NSTGg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-GLQtm2htFIQMtv5rB69Drg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-DZsMIAdx9B3zPWMiPQKiPw.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-lQWnOJ7uNcS3_SSa3dqcgQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Oe86ReWwQVBb7agvroSYHA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Esj_416JZcspzwC4rr94Eg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Mjzkmeik_oLVS9ZIkRYqzQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-__7zl8R1KW4YY0z7R_Ezbw.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-g19JaV0t4TPV3OUd770FIQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-vyekEtd1fR1WvxuV-t9-HA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-okL6Xu2NFT_f4FCitTmawQ.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-ddgD11Glg3iQH2T0qrB5-w.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-JlEtPWq5NLA9Gr4S9IJPZA.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ll58yuEBL5hFiKdkQp0Mjg.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-KAl8Fbd3i99w2ENheIDH4A.jar
Apr 21, 2020 6:47:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ll58yuEBL5hFiKdkQp0Mjg.jar
Apr 21, 2020 6:47:14 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ll58yuEBL5hFiKdkQp0Mjg.jar&uploadType=resumable&upload_id=AAANsUnOTSoE7HTQMUGZBt1V3hYB65MxJQ8IawMmUWnopn5busJ4HDWrMhPoAh80epIB1vhIFyarGg4nwPR1UH57Oi_cS281ig.
Apr 21, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 21, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ll58yuEBL5hFiKdkQp0Mjg.jar
Apr 21, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 11 seconds
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 21, 2020 6:47:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash D3kILbn72HbFHuKKF7Y24Q> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-D3kILbn72HbFHuKKF7Y24Q.pb
Apr 21, 2020 6:47:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 21, 2020 6:47:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-20_23_47_24-10349777963004450754?project=apache-beam-testing
Apr 21, 2020 6:47:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-20_23_47_24-10349777963004450754
Apr 21, 2020 6:47:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-20_23_47_24-10349777963004450754
Apr 21, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T06:47:24.216Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 21, 2020 6:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:27.376Z: Checking permissions granted to controller Service Account.
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:31.916Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.429Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.468Z: Expanding GroupByKey operations into optimizable parts.
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.510Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.583Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.616Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.647Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:32.688Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:33.144Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:47:33.234Z: Starting 5 workers in us-central1-f...
Apr 21, 2020 6:47:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T06:47:42.603Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 21, 2020 6:48:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:02.527Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 6:48:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:17.955Z: Workers have started successfully.
Apr 21, 2020 6:48:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:17.994Z: Workers have started successfully.
Apr 21, 2020 6:48:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:50.924Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 6:48:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:51.128Z: Cleaning up.
Apr 21, 2020 6:48:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:48:51.222Z: Stopping worker pool...
Apr 21, 2020 6:50:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:50:30.844Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 21, 2020 6:50:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T06:50:30.885Z: Worker pool stopped.
Apr 21, 2020 6:50:38 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-20_23_47_24-10349777963004450754 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0aed066f-82f9-47c1-a48a-3c8839d0c1ec and timestamp: 2020-04-21T06:50:38.237000000Z:
Metric: Value:
read_time 15.567
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 36.178 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 39s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/hc3khqh2myg7w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #409
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/409/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-6860] Fix iobase non-global windows bug
[skoonce] [BEAM-9773]: Update Dataflow Debug Capture to use Google API client
[valentyn] Renames run_test -> run_scenario so that nose test runner does not think
[huangry] Fix user distribution metric test by using less restricting value.
[huangry] Apply yapf fix.
[iemejia] [BEAM-9569] Fix BeamSqlPojoExample logRecords method to not require Row
[kawaigin] add display:block to datatable so that the columns can have various
[skoonce] Remove all Dataflow google-api-client-jackson(1) dependencies
[huangry] Replace all explicit validation.
[skoonce] Missed cleanup
[github] [BEAM-9788] Add RetainDockerContainers flag support to Go SDK. (#11467)
[github] [BEAM-9775] Add Go support for SDF StandardRequirements. (#11443)
[github] [BEAM-9496] Add to_dataframe and to_pcollection APIs. (#11264)
------------------------------------------
[...truncated 283.50 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 21, 2020 1:10:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 1:10:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 21, 2020 1:10:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 21, 2020 1:10:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 21, 2020 1:10:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 21, 2020 1:10:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Ll6CvdjZr6iDbIeVoCFR8Q.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-uVVX-KC_-k2y8xwcNVrRKg.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-qVYmvnVekJUEv7XmWR7EWw.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-2Zr50U6TCX9rv8RF1DAXEQ.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-XYNd9L-gW_xXBkCKO2BO6Q.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-tCk2vRkkDI7E5cc80ezOUA.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-C2df-VkB4KDQMYV_UIkfKA.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-vpv_kbmfFWjaYr1VJem9MQ.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-MVqtUc1_gvVYGyUe9b29dw.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-S9BHRj1IvDCX8Eu0Q9ZMfw.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-x0wGHi97u-A0YfrpF-p-fQ.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-KOHyPbI-ng1hPL8XjrE8Qw.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-ZINI_-9BC4VvxA-9tjl6hg.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-492oP7j4Vu8LzvyJB-38ug.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ppWYmgbO4jVfvtn72-p6Ew.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-8s8SgGP4wzfNPgDWLQVEWg.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Sp6nTKCSxudtQ1TFo6Z2Yw.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-m5qkTaxdyj3t6rySNrjMJA.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-vK_mnqOrQkB4FZ629rutrA.jar
Apr 21, 2020 1:10:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-PMMHDte7mvVpWqhBm9DGvw.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ayi09ugFpP-iBeI8paL-Uw.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-nb0709r_8ojv02FNt_21lg.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-5raYBrT3HXJgrmO0iL35VA.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-JlJy3-jWNb3fhhcmTkAajw.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-3VYtOOYHFRjNidcHkLGg4Q.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-rxp7GIqpg35nvGqm14CZ5A.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ayi09ugFpP-iBeI8paL-Uw.jar
Apr 21, 2020 1:10:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-IV62hxwAcRHIt6UVcoh06w.jar
Apr 21, 2020 1:10:41 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ayi09ugFpP-iBeI8paL-Uw.jar&uploadType=resumable&upload_id=AAANsUk2VPNYHkwUz9QnESt2EfcVty5uMSHpkx3PMvZ3PvSJqnYxU_THDlOzv2qtWbzuO_foS-Yj3ByxroNOP8zLiC5puryqGw.
Apr 21, 2020 1:10:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 21, 2020 1:10:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ayi09ugFpP-iBeI8paL-Uw.jar
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 21, 2020 1:10:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 21, 2020 1:10:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 3UPsnw5P2YVGUKlMel3ExQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3UPsnw5P2YVGUKlMel3ExQ.pb
Apr 21, 2020 1:10:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 21, 2020 1:10:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-20_18_10_48-12075377291359085889?project=apache-beam-testing
Apr 21, 2020 1:10:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-20_18_10_48-12075377291359085889
Apr 21, 2020 1:10:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-20_18_10_48-12075377291359085889
Apr 21, 2020 1:10:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T01:10:48.765Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 21, 2020 1:10:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:53.787Z: Checking permissions granted to controller Service Account.
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.026Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.797Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.830Z: Expanding GroupByKey operations into optimizable parts.
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.863Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.949Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:58.978Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:59.009Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 21, 2020 1:10:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:59.040Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 21, 2020 1:11:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:59.432Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 1:11:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:10:59.503Z: Starting 5 workers in us-central1-c...
Apr 21, 2020 1:11:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-21T01:11:04.582Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 21, 2020 1:11:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:11:33.042Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 1:11:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:11:33.071Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 21, 2020 1:11:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:11:39.065Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 21, 2020 1:11:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:11:53.534Z: Workers have started successfully.
Apr 21, 2020 1:11:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:11:53.568Z: Workers have started successfully.
Apr 21, 2020 1:12:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:12:35.272Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 21, 2020 1:12:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:12:35.492Z: Cleaning up.
Apr 21, 2020 1:12:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:12:35.574Z: Stopping worker pool...
Apr 21, 2020 1:14:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:14:01.961Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 21, 2020 1:14:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-21T01:14:02.010Z: Worker pool stopped.
Apr 21, 2020 1:14:07 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-20_18_10_48-12075377291359085889 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f37a7487-a723-434a-bd63-839a6ed5b0b1 and timestamp: 2020-04-21T01:14:08.030000000Z:
Metric: Value:
read_time 19.253
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 38.427 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 50s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/exkpzv2xglvue
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #408
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/408/display/redirect?page=changes>
Changes:
[michal.walenia] [BEAM-9147] Add @Experimental annotation to VideoIntelligence related
[michal.walenia] Add @Experimental to the package-info as well
[github] Add release date for 2.20.0
[github] [BEAM-9692]: Make CombineValues portable (#11335)
------------------------------------------
[...truncated 285.16 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 20, 2020 7:01:05 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 20, 2020 7:01:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 20, 2020 7:01:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-3oCzimHXVGVH-CHsngQM4g.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-nfZ1ABEP2khvB4DI0O6mNw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Db7-JbNL5RAXqKbfphfHPg.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-6RnZbu8YhpJ4m35JJ8igiA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-_HeR5zyydgkleoCqvSkvEQ.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Pghi3L290VatTmeyj2z3Sw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-t94SPPPNsPcG5ztfFfTjUA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-sg2xs6sS683zbhqMBe9jjA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-qxMC9s-CMBY8R3gLE7hTrQ.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-KgnputiUi1XgZkZeWjXnOw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-vJyzEjZzCxEq3YVO7fzddg.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-peGJio-O9W-BIG8_p5VH-A.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-UoWn05mVxQ_6s8pOZ4PXNw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-UcXV-EBcZvX8Qu_3MaoUEA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-FhGcthVDBtE11Od54-3LWA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-GVuukIeM84YusahW8oNcWA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-rLR56H_g3FhZuDQkxXsYzA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-5oKh22JvQ3cOOdD58c3qDA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-LMUH7QWfgJ39gtWvavBojw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Oi3jDjJ4xniEYuGK8amu0Q.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-pFtQmub0ah_6bag8mWeTxg.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-gdljSdAh9MP-f6rmUXyEoA.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-qDn_K7FEDbF7e2yp_MFygQ.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ON53CURTAAsPVfDAXypH2Q.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-mQgPPyYvqn8iRdUsER055g.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-wLn8l2_ze6vZqTrb9isUNw.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-deK3KK-DxcpSOs2mAL6G6w.jar
Apr 20, 2020 7:01:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-deK3KK-DxcpSOs2mAL6G6w.jar
Apr 20, 2020 7:01:09 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-deK3KK-DxcpSOs2mAL6G6w.jar&uploadType=resumable&upload_id=AAANsUkPC6q7Q-oKggURg3XHhwz1O63yuPBDKBj9xLL7b6vDnTttcMPSiT3v0leeTomMob07uYXCH5Hx4ZmDo6w1YPBnD8rc5A.
Apr 20, 2020 7:01:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 20, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-deK3KK-DxcpSOs2mAL6G6w.jar
Apr 20, 2020 7:01:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 20, 2020 7:01:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1--1rdXBhaXVFljcLOfgKg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1--1rdXBhaXVFljcLOfgKg.pb
Apr 20, 2020 7:01:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 20, 2020 7:01:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-20_12_01_15-6678779944365729387?project=apache-beam-testing
Apr 20, 2020 7:01:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-20_12_01_15-6678779944365729387
Apr 20, 2020 7:01:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-20_12_01_15-6678779944365729387
Apr 20, 2020 7:01:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T19:01:15.128Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 20, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:18.930Z: Checking permissions granted to controller Service Account.
Apr 20, 2020 7:01:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:22.674Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.262Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.302Z: Expanding GroupByKey operations into optimizable parts.
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.329Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.401Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.427Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.454Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.488Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.818Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 7:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:23.885Z: Starting 5 workers in us-central1-f...
Apr 20, 2020 7:01:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:50.381Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 7:01:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:50.406Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 20, 2020 7:01:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:55.769Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 7:01:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:01:55.802Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 20, 2020 7:01:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T19:01:58.514Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 20, 2020 7:02:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:01.193Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 7:02:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:13.364Z: Workers have started successfully.
Apr 20, 2020 7:02:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:13.395Z: Workers have started successfully.
Apr 20, 2020 7:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:47.971Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 7:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:48.211Z: Cleaning up.
Apr 20, 2020 7:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:02:48.306Z: Stopping worker pool...
Apr 20, 2020 7:05:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:05:02.902Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 20, 2020 7:05:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T19:05:02.941Z: Worker pool stopped.
Apr 20, 2020 7:05:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-20_12_01_15-6678779944365729387 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b945b00c-d538-4c0c-84f1-14147890dca6 and timestamp: 2020-04-20T19:05:07.990000000Z:
Metric: Value:
read_time 14.679
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.038 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 4 mins 13.663 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 29s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/jbzvnrfry4tge
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #407
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/407/display/redirect>
Changes:
------------------------------------------
[...truncated 283.33 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 20, 2020 12:51:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 20, 2020 12:51:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 20, 2020 12:51:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-clzwk27Xefo8OzCPtbcQ-w.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Qmdv_B02L98InvM8plCaXQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-43SnupFj4aS4waiWPYol6g.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-_kh_OGc7kulMNhPNzKz16A.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-DkZiD7giBSsYxgryI1KiFQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-OjHC5yvXNX_4NspHgpBslA.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-CIM8UUopyESB-cbnfIVxqA.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TgspMNHnHHD4nrriMnk5nQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-MtcVxesf0THEvMII7tfAzQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-MufE8n7QwbRaOQN_9Q6nGw.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Dzbu5yKVhQPc5J9exPoJoA.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT--1VX-Fhdf7eNMdSSsWlBag.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-47dpzzAj7Lnes1V-mlpcOA.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-4Fy35vMAYLfc4Hc-ZCzz3Q.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-e3-IZZ3LTTQ-iY6VuoK9LQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-F8ieknZSLB0diFGu6Erh3A.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Etjs1ez77BTuAgmhxsU2ZQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-IMwgdtxpzrOnHhJSVrDFOQ.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-tSkyEs-2Up_BIpIgrs0xcg.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Qigs-wtKL7M-vr5zXz2oUA.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-7L9101jhXLYadJKP8Mk-aw.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-fKkaHezLJtKXJ8CpxB1_8w.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-StFSDKgIrSeEuwJSqXFvmg.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-xwpAjJgPj-IOdSiPnGaFTw.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ap5Yy5DHM3Rcz-NQ7i8-Pw.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ap5Yy5DHM3Rcz-NQ7i8-Pw.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-k_G-7cc7kDlD80Vkb33aLg.jar
Apr 20, 2020 12:51:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-fC_s0Ar-Vkzu0N9IQfa8xg.jar
Apr 20, 2020 12:51:19 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ap5Yy5DHM3Rcz-NQ7i8-Pw.jar&uploadType=resumable&upload_id=AAANsUk2a04sE7UFUGSRfWcnRH1P4C73OunZn7EXHc_9BqSeGWzcUrTyeWvBxyDYP--c_2s_9eiyjtbxdQ9dFYNcdN0X2CEyLA.
Apr 20, 2020 12:51:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 20, 2020 12:51:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ap5Yy5DHM3Rcz-NQ7i8-Pw.jar
Apr 20, 2020 12:51:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 20, 2020 12:51:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash bIJn3d089HCshepEmkKrVg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bIJn3d089HCshepEmkKrVg.pb
Apr 20, 2020 12:51:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 20, 2020 12:51:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-20_05_51_24-2509410074011457398?project=apache-beam-testing
Apr 20, 2020 12:51:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-20_05_51_24-2509410074011457398
Apr 20, 2020 12:51:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-20_05_51_24-2509410074011457398
Apr 20, 2020 12:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T12:51:24.818Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 20, 2020 12:51:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:28.349Z: Checking permissions granted to controller Service Account.
Apr 20, 2020 12:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:31.606Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.204Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.245Z: Expanding GroupByKey operations into optimizable parts.
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.279Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.354Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.393Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.417Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.453Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:32.876Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 12:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:51:33.027Z: Starting 5 workers in us-central1-f...
Apr 20, 2020 12:52:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T12:52:03.848Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 20, 2020 12:52:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:07.363Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 12:52:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:07.403Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 20, 2020 12:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:12.809Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 12:52:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:25.524Z: Workers have started successfully.
Apr 20, 2020 12:52:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:25.565Z: Workers have started successfully.
Apr 20, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:57.794Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:57.992Z: Cleaning up.
Apr 20, 2020 12:53:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:52:58.077Z: Stopping worker pool...
Apr 20, 2020 12:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:55:01.526Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 20, 2020 12:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T12:55:01.569Z: Worker pool stopped.
Apr 20, 2020 12:55:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-20_05_51_24-2509410074011457398 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c4fa2802-c630-433f-8b9b-5dea99c93c6c and timestamp: 2020-04-20T12:55:08.593000000Z:
Metric: Value:
read_time 17.861
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 0.792 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 5s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/qw2t7flkyhl34
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #406
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/406/display/redirect>
Changes:
------------------------------------------
[...truncated 282.76 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 20, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 20, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 6:45:09 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 20, 2020 6:45:10 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 20, 2020 6:45:10 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 20, 2020 6:45:10 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 20, 2020 6:45:10 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-DMxjBmQmOP6Xw9YCP7DJiQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-MpHdRk1CEhaWZ6kAWLvhig.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-XAuqMxni_kv1E1cTnLdAsQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-JneOzZOhBO8PkbhMMmkO4g.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-MGNEUT7ct3uPrEnlo9cEQA.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Ycm-NppVb8INa3NjzrsEdA.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-LzkWHQO8_YDNwFZ3-bjDVQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-Dm1aKLQ02nWh__vp8DjE2w.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-x4uEf2IFe2JAY1zS0h6lwg.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-YWi3hmYVt0oNvMFAOd-R5w.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-f-F9ajlSMt_M5T7kNi0xpg.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-xoQIJSoHeYV54SeqJJ0vJg.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Ly9BxYbCI-aaxk19ikyUnw.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6gd8lDR-c4ltqeSNMz1UfA.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-g2vF2kmcLCSiSwGEGRG9IQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-zaVGtbzxJBBw1XNzoCGwZw.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-lzg_OAL64r1wDuTK4QfScQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-EnZ0hfzMGLi-BufcfpYnmw.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-eD_O_yexU7hC1RgEKQ6lpw.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-R0aN6rVDoWwwgqXeqi_oMA.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-z-WIn_B-vfUA6Csw2o5hjQ.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-OvC5_NsgPM9xOM_NPD_x8g.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-NgJMm1JcZgjg9SfFNEFSfA.jar
Apr 20, 2020 6:45:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-Hwienr_gQiQw6vDSs0aVEA.jar
Apr 20, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-nFLGjQR2_8MIDLeh-2JlcA.jar
Apr 20, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-u1gIYALwIz1uHmp1OUV7ZA.jar
Apr 20, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m0BBhbjzZX5jHlrIRY8mWw.jar
Apr 20, 2020 6:45:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m0BBhbjzZX5jHlrIRY8mWw.jar
Apr 20, 2020 6:45:13 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m0BBhbjzZX5jHlrIRY8mWw.jar&uploadType=resumable&upload_id=AAANsUl0Q9GVspGXWB_sB8jH0r1h7N8dvnFw0HQLvqXJBaXEh82dH7OG3iKDMDfzqpizKilwIkJjcpbXR0AVt1eh_aE.
Apr 20, 2020 6:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 20, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-m0BBhbjzZX5jHlrIRY8mWw.jar
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 20, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash cDaDhsGd40XdIX41oBEDig> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cDaDhsGd40XdIX41oBEDig.pb
Apr 20, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 20, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-19_23_45_20-7887466430531156144?project=apache-beam-testing
Apr 20, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-19_23_45_20-7887466430531156144
Apr 20, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-19_23_45_20-7887466430531156144
Apr 20, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T06:45:20.265Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 20, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:23.274Z: Checking permissions granted to controller Service Account.
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:27.452Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.049Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.101Z: Expanding GroupByKey operations into optimizable parts.
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.133Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.222Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.253Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.292Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 20, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.326Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 20, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.706Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:28.778Z: Starting 5 workers in us-central1-f...
Apr 20, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T06:45:39.557Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 20, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:45:57.526Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:46:13.667Z: Workers have started successfully.
Apr 20, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:46:13.696Z: Workers have started successfully.
Apr 20, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:46:45.327Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:46:45.619Z: Cleaning up.
Apr 20, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:46:45.699Z: Stopping worker pool...
Apr 20, 2020 6:48:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:48:24.235Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 20, 2020 6:48:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T06:48:24.280Z: Worker pool stopped.
Apr 20, 2020 6:48:31 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-19_23_45_20-7887466430531156144 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3c3e1efe-d1cb-46c4-9edd-757c62cdacb6 and timestamp: 2020-04-20T06:48:31.856000000Z:
Metric: Value:
read_time 12.188
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 30.535 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 14s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/5zvi2gn25ho2u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #405
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/405/display/redirect>
Changes:
------------------------------------------
[...truncated 283.37 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 20, 2020 12:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 20, 2020 12:45:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 20, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 20, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-tKUz8xrDtb3Cwfso5z1-Gw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-rYcz7IzMRCSQVWJ4fwx-IQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-4OVRnbQJG2Ux3mgsbHr6Aw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-IbVfiCabH5udHsX1NLz1BQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ny2TCR-3bPYFf4xcAUQJjw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-9KkP26Kobwmgpdqpshysew.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-_eNbGdAViGBPO1WjYTAEAQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-cF_OjM3qzNMeUBinORsHvw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-bGSF-kjvZ9lRz6lfPsbtpw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-yXWvHgAmYx46vVcqqDsfcg.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-UZ63UiZ17X8HjYjBnl6-Og.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-rhjnuCd6sDyTyY8t0ej6vQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-FVfw7qHI2rsngrjuRjYZaQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Zm072peWD6NptJ-AOSOYag.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-jZLqJw3Nfx7BAsmWzsHgCQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-H2ZL6_qP9uYEWgTFMBp7Ag.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-u_1j6ZJlgbrJ-ebZT5pYnA.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-UupRm5ivY9TiQFHaPPmOKw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Gq1usMGuqzSS9LXM1fehEQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-8ntwzg-8289vvrx4kWVQXQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-FBC99WYb_PCGqmEKU623KA.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-_xmlmaCXjLPaC7RjVyRSKA.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-UjZUZ5bvfwqgVTj-uqL4cA.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-oHCzTulbarVDVj7YruG5OQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-nXwL845tIaFEcgWYm_rksw.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4LU11rQ7jftTu3bRPhwFgQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4LU11rQ7jftTu3bRPhwFgQ.jar
Apr 20, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-bREMv3yP10v_b4GzlXtveA.jar
Apr 20, 2020 12:45:19 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4LU11rQ7jftTu3bRPhwFgQ.jar&uploadType=resumable&upload_id=AAANsUlB6Uxat6arYGJbFSFiswLzXyP3qy1SbiLdqvaXmlbDKM-T03AmRQj-aS2Q0lieKOzyj29Jqb4yF9qYFibbgfw.
Apr 20, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 20, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4LU11rQ7jftTu3bRPhwFgQ.jar
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 20, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash l4uqTt8TTlcTrMVl21f9wA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-l4uqTt8TTlcTrMVl21f9wA.pb
Apr 20, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 20, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-19_17_45_26-16442932288374089888?project=apache-beam-testing
Apr 20, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-19_17_45_26-16442932288374089888
Apr 20, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-19_17_45_26-16442932288374089888
Apr 20, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T00:45:26.311Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 20, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:30.202Z: Checking permissions granted to controller Service Account.
Apr 20, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:34.244Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:34.831Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:34.872Z: Expanding GroupByKey operations into optimizable parts.
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:34.903Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:34.992Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:35.025Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:35.060Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:35.097Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:35.499Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:45:35.584Z: Starting 5 workers in us-central1-f...
Apr 20, 2020 12:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:00.228Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 20, 2020 12:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-20T00:46:08.480Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 20, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:19.443Z: Workers have started successfully.
Apr 20, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:19.477Z: Workers have started successfully.
Apr 20, 2020 12:46:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:50.507Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 20, 2020 12:46:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:50.703Z: Cleaning up.
Apr 20, 2020 12:46:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:46:50.775Z: Stopping worker pool...
Apr 20, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:48:33.727Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 20, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-20T00:48:33.776Z: Worker pool stopped.
Apr 20, 2020 12:48:41 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-19_17_45_26-16442932288374089888 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8a2eae05-3316-4840-a31d-759884953c8f and timestamp: 2020-04-20T00:48:41.094000000Z:
Metric: Value:
read_time 13.459
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 34.903 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 23s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/gebpcavvfptmk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #404
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/404/display/redirect>
Changes:
------------------------------------------
[...truncated 282.42 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 19, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 19, 2020 6:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 19, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-AibKDb8AkXRTlEtltwNNAg.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-BzAzT-EvJIA9UrBDS_JP1Q.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-o9skuTSTagnLzUE75eqoug.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-WObngZ6cyj1QwXwcwoqefA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Y4O5oNKAd4jKUWSxb04-QA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-IKuUnNoUqfQCpn-DDMxLnA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-oDDhkfsHm-d0QQkiw-R5dA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-4dhdOMuu9kANVmnMannLSw.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-rYZHWQTQ39zj0MlaOFkdag.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-4eh_u3uUmywSBMMzTFXScQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-qE0KmqA4LSKV9J8TGrHyAw.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-EV9icuUTZtOJ221hcx3n4A.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Z23S6JR81ICHR9m9QcrSag.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-6zSri9YVg565cov9ZQ4BUA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-i5CBB5X5xsU-EiUuf1NLtQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-643ROfEnfSa6mAESZqkGqw.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-XPVW9fEB2mNjuVG4NsLhVg.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-w89oByKfROhnSQynly8VWA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-P-3j5s3kND_ky7bxhUa1jQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-DiR1uY2mObKzKtx_4OYDVQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-HF6QQOaUIkJevwQAR8o4XQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-T8AInOhKvgSgaGDY3QpJYw.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4uKp1WMRLRXFryGCLDkM6w.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7NV0JhfzToItIJxAvTm_GA.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-MdFvHffd1bNZB8fl5-ZYuw.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-RVSJvtSKAvZbWd0YRHllDQ.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-vKPqFKXMxsndXvsxY8Vf4g.jar
Apr 19, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7NV0JhfzToItIJxAvTm_GA.jar
Apr 19, 2020 6:45:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7NV0JhfzToItIJxAvTm_GA.jar&uploadType=resumable&upload_id=AAANsUmnBqTevugRGG5XHICPzWOab6uQSyR6AuWEsa16HEZNvbz2fzwy_SCjiY5JhP34dOF4fsIcWvjNpOTN9m2ItMEcG-BYqA.
Apr 19, 2020 6:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 19, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-7NV0JhfzToItIJxAvTm_GA.jar
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 19, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1vIVX8PDySMijk2enzcO8w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1vIVX8PDySMijk2enzcO8w.pb
Apr 19, 2020 6:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 19, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-19_11_45_39-15111030178277706174?project=apache-beam-testing
Apr 19, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-19_11_45_39-15111030178277706174
Apr 19, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-19_11_45_39-15111030178277706174
Apr 19, 2020 6:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T18:45:39.302Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 19, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:42.099Z: Checking permissions granted to controller Service Account.
Apr 19, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:45.941Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.726Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.777Z: Expanding GroupByKey operations into optimizable parts.
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.812Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.885Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.910Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.944Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:46.974Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:47.428Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:45:47.510Z: Starting 5 workers in us-central1-f...
Apr 19, 2020 6:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T18:46:12.724Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 19, 2020 6:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:46:13.012Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 6:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:46:30.256Z: Workers have started successfully.
Apr 19, 2020 6:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:46:30.287Z: Workers have started successfully.
Apr 19, 2020 6:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:47:02.366Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 6:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:47:02.766Z: Cleaning up.
Apr 19, 2020 6:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:47:02.849Z: Stopping worker pool...
Apr 19, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:48:38.387Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 19, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T18:48:38.425Z: Worker pool stopped.
Apr 19, 2020 6:48:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-19_11_45_39-15111030178277706174 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 67d8fb24-4006-474b-b4b8-ec27849bc57d and timestamp: 2020-04-19T18:48:45.032000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.759
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 24.633 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7uuvwdnsgokgg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #403
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/403/display/redirect>
Changes:
------------------------------------------
[...truncated 283.70 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 19, 2020 12:51:42 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 12:51:42 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 19, 2020 12:51:42 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 12:51:42 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 19, 2020 12:51:43 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 19, 2020 12:51:43 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 19, 2020 12:51:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 19, 2020 12:51:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-h7qOypM6PW22MObvbsTKgg.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-IfctNNDzt1UxOVjg-PB8VQ.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-L_xqxdEm7rP-JEMtfoBFFw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Qb3E0kkzo06KLF5kDFhY5w.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-LIfjZ4pPL1r0j366J_rghQ.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-nh49eG00NWQdmxgTkdGZRA.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-RlVKXhtp6ebu_SdPaWB_CA.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-isnNCvAAxtitA5N3zddfsw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-GGsOOpN5xIPyrwuibkgv2w.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-elaqn1jBZB-5y21vuhd9OQ.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-G46RsV_StlYIJfjIfT5T4A.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-fn8II7IkG6YgR8AHoE2W4w.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-c3Pgs_uMaSQ3fePqDbGZIA.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-_Rm8b_MddtAu7_Wwp3K1Gw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-RgPNilLIPDoQRlsBoDMgCA.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-W_Mp-4LJ29--h9Gksfjjzg.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-DMGyFfeKi67OxRENKL6ZlQ.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-9IxshpkWaLhTBzcYt-xllw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Ux5avFFgghv7-dIlTO4OqA.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-J_gD_jzRUz9jqgK3A2UiWw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-r0u3viRiMzl3aNhB3Mc12w.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-NiXXSS5wsqanDMIX5aBCFQ.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-oTG70SIjI9mH094Fe1kXrw.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-BJUcw_Vdlrxc6t28aF1BHg.jar
Apr 19, 2020 12:51:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-_5ZVpKbXLUkp1ocyz4zXgA.jar
Apr 19, 2020 12:51:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ABaViDuiO1CSd-0HxJ4g2A.jar
Apr 19, 2020 12:51:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kTUeIuRxZTECJkKmd5y0Jg.jar
Apr 19, 2020 12:51:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kTUeIuRxZTECJkKmd5y0Jg.jar
Apr 19, 2020 12:51:46 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kTUeIuRxZTECJkKmd5y0Jg.jar&uploadType=resumable&upload_id=AAANsUnYBF5xV-Qbg3_PJ0CcSW1sW4j-EwUGjiFR6mDFMDrHN115F2eZGHXd58NXKJMfLkLxDqAVfBsktyZ5dhYPIz4J99o5hw.
Apr 19, 2020 12:51:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 19, 2020 12:51:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-kTUeIuRxZTECJkKmd5y0Jg.jar
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 19, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash gVC5HioQaIX0qd5xPRSG_g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gVC5HioQaIX0qd5xPRSG_g.pb
Apr 19, 2020 12:51:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 19, 2020 12:51:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-19_05_51_53-12350733304072602504?project=apache-beam-testing
Apr 19, 2020 12:51:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-19_05_51_53-12350733304072602504
Apr 19, 2020 12:51:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-19_05_51_53-12350733304072602504
Apr 19, 2020 12:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T12:51:53.168Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 19, 2020 12:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:51:56.341Z: Checking permissions granted to controller Service Account.
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:00.345Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:00.998Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.029Z: Expanding GroupByKey operations into optimizable parts.
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.066Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.153Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.193Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.218Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 19, 2020 12:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.253Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 19, 2020 12:52:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.634Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 12:52:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:01.709Z: Starting 5 workers in us-central1-f...
Apr 19, 2020 12:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T12:52:18.279Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 19, 2020 12:52:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:28.646Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:28.680Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 19, 2020 12:52:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:34.110Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 12:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:46.112Z: Workers have started successfully.
Apr 19, 2020 12:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:52:46.153Z: Workers have started successfully.
Apr 19, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:53:21.531Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:53:21.740Z: Cleaning up.
Apr 19, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:53:21.813Z: Stopping worker pool...
Apr 19, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:54:58.804Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 19, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T12:54:58.839Z: Worker pool stopped.
Apr 19, 2020 12:55:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-19_05_51_53-12350733304072602504 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f59e0dca-f62b-4114-ae2a-085268d1becc and timestamp: 2020-04-19T12:55:06.069000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.392
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 31.453 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 34s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/25afzjdktuiyy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #402
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/402/display/redirect>
Changes:
------------------------------------------
[...truncated 283.26 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 19, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 19, 2020 6:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 19, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-G_YB3FIvuu_WVL3vkjx8lw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-uoEXbC3s28sym6rNxo2Vdg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-RcQMQdL9AQRqq5AJpkBJWw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-GpXut-SU6g6Sf-mdfY4Puw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-FPztYFjOouHYfq1vrEdyWg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-B8gHPQynsls_07O6_uSJcg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-JLGa7BbYVTHrmirnNyIJEA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-usRdMpucTh8td29-nqV2Aw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-W0xHgEr_VnBgP48lylVcvw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-uEJJfZ5jPhT9X52-53XKpA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-zV6nauPaHp75mTOkzGE1yw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-PesVKgFsC1bsBSLHZgjRnw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-ZBOD6ouvvkhXLymt0OzgbQ.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-10fmCKceLHOmn-ziy_7Dpg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-9Ur3EzD5i8U2ONvwl-WPVA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests--y5zmUv6ydquGtw49tFcHg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-7Ouvhb4vHBzhn_3C2m1-BA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Dd6SOeRxbYq4cfCNmP-bmA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-gHR1GqBQyHoIY2odZvq6pw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-4-RxBVdWypxvtcSBVcWkBw.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-RKadKoa0VWwd3K2dtBWqzg.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-rctdcaldnMDTrGr9ApmoeQ.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-tQCsziDktPiu4BDUHchw8A.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-BG0CV6ye3_8iZzjX92o4wA.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-8cV-8gW8y0srg3Odz8av0A.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-tQiKbA5rupvBCu9FjZkKMQ.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8v6KmG2_CuebQBzQXmaG1Q.jar
Apr 19, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8v6KmG2_CuebQBzQXmaG1Q.jar
Apr 19, 2020 6:45:19 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8v6KmG2_CuebQBzQXmaG1Q.jar&uploadType=resumable&upload_id=AEnB2Ur8H91VrUTCJiZdySeMOd-aLGBqD4e97VuCz6BKRZPsw6AE7aEXMgj5l67YDOizLjXu5TzRZTtBJ6xSnAt5lqvT9-IJE6ewT4j1x0O4UpzaA4_aShI.
Apr 19, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 19, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8v6KmG2_CuebQBzQXmaG1Q.jar
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 19, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 19, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash zqfibucLIC872cYFyfyCfA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-zqfibucLIC872cYFyfyCfA.pb
Apr 19, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 19, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-18_23_45_29-1533946369346600827?project=apache-beam-testing
Apr 19, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-18_23_45_29-1533946369346600827
Apr 19, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-18_23_45_29-1533946369346600827
Apr 19, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T06:45:29.673Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 19, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:33.345Z: Checking permissions granted to controller Service Account.
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:36.862Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.514Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.555Z: Expanding GroupByKey operations into optimizable parts.
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.587Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.662Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.700Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.728Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:37.760Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 19, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:38.194Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:45:38.344Z: Starting 5 workers in us-central1-f...
Apr 19, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T06:45:44.711Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 19, 2020 6:46:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:04.182Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 6:46:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:04.220Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 19, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:09.581Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:26.659Z: Workers have started successfully.
Apr 19, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:26.691Z: Workers have started successfully.
Apr 19, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:57.465Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:57.656Z: Cleaning up.
Apr 19, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:46:57.730Z: Stopping worker pool...
Apr 19, 2020 6:48:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:48:41.687Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 19, 2020 6:48:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T06:48:41.735Z: Worker pool stopped.
Apr 19, 2020 6:48:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-18_23_45_29-1533946369346600827 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): df45a4f1-a9ec-4223-9c72-660336119ccd and timestamp: 2020-04-19T06:48:49.164000000Z:
Metric: Value:
read_time 13.728
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.042 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.055 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 41.661 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 32s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/l24cmaimdjyfy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #401
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/401/display/redirect>
Changes:
------------------------------------------
[...truncated 283.34 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 19, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 19, 2020 12:45:13 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 19, 2020 12:45:13 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-vUbHbNnRgr2h-3boKhKmlg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-MovLUZlkUh9X8T2vCX1fYA.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-C2cAlaYTctpeGXOFezo2sg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-xqpSUtyV2mMWWKPzdmaXWg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-5uJa1fy6UUySUMTR2-Q4_Q.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6W-rVTwdr30Lv79H4-3keQ.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-E6JzZhsdL0y57qhXc_7cpA.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-dVyklrgukxbPL0SKSx-Tvw.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-FQ5zs4J_XgnpWKkm9vwdGg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-JSOfivZSp_3Oc6-YWUZItQ.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT--Ez2Am3EERrAMcvQH8qJ3A.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-nFEaP7dHyaYbUse6kBIqCg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-IpqI_v06wUY_drih-GVsSg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-5aVerPW0Wx90un0KpCxj3g.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-sgPgHJOtcHkYQ4lcUVDqjQ.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-N-F8af7j4nE1TdC-gcTy8A.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-LHrwnIXaThhubnRe4xeXkg.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-cx3WAW446SHu0bDv2MHhpw.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-TPixn5_glE1fpkUzVzhwPw.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-emvkMkIRC2D85lAUDAb7Kw.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded--GHJC8YFzhH3GXTw-f-pvA.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-P1t5WIG0mEdFSNo3ZUj1rA.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-kB4Yxje_TpeiAlkek8lbfQ.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-scE5N6h6dx6OSvhYxgYKIQ.jar
Apr 19, 2020 12:45:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-IaKuUOeRrKDadSas7ay3Kg.jar
Apr 19, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-DUTZM6DLFbMnhPJgWs9g6g.jar
Apr 19, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-vTou61kHDYcYKqrvEAL7xg.jar
Apr 19, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-DUTZM6DLFbMnhPJgWs9g6g.jar
Apr 19, 2020 12:45:16 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-DUTZM6DLFbMnhPJgWs9g6g.jar&uploadType=resumable&upload_id=AEnB2UpJXbbchdcp_ZaEwdrFoLFxX8-okjtS6Kwn6SdtcegQ6-ey59A8VoIJfRiW77aCT0mpbLt-ctnAds6p282fL2GgNXukng.
Apr 19, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 19, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-DUTZM6DLFbMnhPJgWs9g6g.jar
Apr 19, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 10 seconds
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 19, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash GoRcR8rae1uvLC3jriZ-nA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-GoRcR8rae1uvLC3jriZ-nA.pb
Apr 19, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 19, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-18_17_45_26-15046755581135730773?project=apache-beam-testing
Apr 19, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-18_17_45_26-15046755581135730773
Apr 19, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-18_17_45_26-15046755581135730773
Apr 19, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T00:45:26.089Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 19, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:29.169Z: Checking permissions granted to controller Service Account.
Apr 19, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:33.427Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 19, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.116Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 19, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.156Z: Expanding GroupByKey operations into optimizable parts.
Apr 19, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.188Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 19, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.280Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 19, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.311Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 19, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.343Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 19, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.374Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 19, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.752Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:34.859Z: Starting 5 workers in us-central1-c...
Apr 19, 2020 12:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:45:59.198Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 19, 2020 12:46:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-19T00:46:04.750Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 19, 2020 12:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:46:18.321Z: Workers have started successfully.
Apr 19, 2020 12:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:46:18.362Z: Workers have started successfully.
Apr 19, 2020 12:46:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:46:54.650Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 19, 2020 12:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:46:54.834Z: Cleaning up.
Apr 19, 2020 12:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:46:54.917Z: Stopping worker pool...
Apr 19, 2020 12:47:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:47:58.369Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 19, 2020 12:47:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-19T00:47:58.416Z: Worker pool stopped.
Apr 19, 2020 12:48:10 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-18_17_45_26-15046755581135730773 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f7d7d160-7e88-4e4a-861c-72e4212fccee and timestamp: 2020-04-19T00:48:10.073000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.768
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 6.036 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 53s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/tgbbxriurpukk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #400
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/400/display/redirect>
Changes:
------------------------------------------
[...truncated 284.51 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 18, 2020 6:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 6:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 18, 2020 6:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 6:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 18, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 18, 2020 6:45:28 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 18, 2020 6:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 18, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-uQ63weR4X6ygEo9TK0ULqw.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-iwi9d1HNm3IyEE_WceqdCg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Vuz_2WGcWkEyR123JD5XMA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-AkURXewNFFq3FWWd2ZFjSg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wbXKuFpe55nMJQgkHSyC1A.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Ov-0iK06MK1E-1Zjqt7Ynw.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-katBisWFfnvoDqHyzPCluQ.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LTMdkh8H8DBtiFGemayVAg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-sIU8upGDAxi6Nh9jNrIVgA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-AYDCgO-ZUF6KZkeDOcviIg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-fJKOdoa3Nuo0YdIrrazxHg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-5JEAZlmFPLlLsKhieI3OgA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-bKrVVz43dKjEzBZ5SDcPtQ.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-jI8FHrchSMK09OviTkPxSw.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-ogPeUiU8veGQTcjCwcsdNg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Iq0shIPCMfIirV-aKeQ6Tg.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-YOqRxdVfP9MbWV_AhqKpbA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ISa4Ku02QwiVTuXgcz5XCQ.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-CZ1gZnCIsNXjFxSpegh8zA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-LGrWotrQBECxhZe5WvryBw.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT--bLwYpoeqEwPx3OCD9vMwA.jar
Apr 18, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-aJyvSY0KrK7eRhIMx4qdtg.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Dg2KKICy3FRu0wVviTFTkQ.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-fLIp6x2wx9H3JD7kYcwjLQ.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-ATyEaITTPujalJzV0pW8FA.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-m39w3GYSK-yZORWsz_h9jA.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Dg2KKICy3FRu0wVviTFTkQ.jar
Apr 18, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-9Ex24fzpsLevSwu6pX9V-w.jar
Apr 18, 2020 6:45:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Dg2KKICy3FRu0wVviTFTkQ.jar&uploadType=resumable&upload_id=AEnB2UqCGlra80jcMRklgiuidXtZYh6uliLdeaIfAgNw5Pz-UYCks7tIf23GhVu-5xJOcgpkp5OoNopFxX7PR_uO8BUU-ChRdYyKU4ssH8dMiP9TKucQCkg.
Apr 18, 2020 6:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 18, 2020 6:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Dg2KKICy3FRu0wVviTFTkQ.jar
Apr 18, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 18, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash rM7ObKknkzETc83eUyfg7A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rM7ObKknkzETc83eUyfg7A.pb
Apr 18, 2020 6:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 18, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-18_11_45_39-8315494645122879565?project=apache-beam-testing
Apr 18, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-18_11_45_39-8315494645122879565
Apr 18, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-18_11_45_39-8315494645122879565
Apr 18, 2020 6:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T18:45:39.113Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 18, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:42.634Z: Checking permissions granted to controller Service Account.
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.095Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.810Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.849Z: Expanding GroupByKey operations into optimizable parts.
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.875Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.956Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:47.990Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:48.024Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:48.063Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:48.579Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 6:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:45:48.663Z: Starting 5 workers in us-central1-c...
Apr 18, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T18:46:03.802Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 18, 2020 6:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:21.596Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 6:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:21.627Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 18, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:43.084Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:43.117Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 18, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:43.393Z: Workers have started successfully.
Apr 18, 2020 6:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:43.430Z: Workers have started successfully.
Apr 18, 2020 6:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:46:48.514Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:47:25.649Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:47:25.846Z: Cleaning up.
Apr 18, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:47:25.936Z: Stopping worker pool...
Apr 18, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:48:37.469Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 18, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T18:48:37.524Z: Worker pool stopped.
Apr 18, 2020 6:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-18_11_45_39-8315494645122879565 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d32f2cb8-790a-4194-a5fd-848c84cf134e and timestamp: 2020-04-18T18:48:43.878000000Z:
Metric: Value:
read_time 19.071
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 24.183 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 27s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/a3escdulwbycw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #399
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/399/display/redirect>
Changes:
------------------------------------------
[...truncated 282.63 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 18, 2020 12:48:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 12:48:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 18, 2020 12:48:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 12:48:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 18, 2020 12:48:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 18, 2020 12:48:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 18, 2020 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 18, 2020 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-3wlU_tZjt1b1lf5Ux26vtg.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-998xwOGB1HypMiNhVNzfgg.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-kfyWo8LC3BCQXTRvNaDExQ.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-__l5jr9T73LFCRRO3_xZXQ.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-cjboVKQ4NRPN-5UeJ_7rng.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-7ahdMGTBW83sXNJ5PRGhwg.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-b5dR-c9GhIjdHQsN_YrBzA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-bfVqEVb7lAmsfiM38-yYog.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wF5SjNl6XYnR3jHF_jmcHQ.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-4Iyt-S7YD6t-B7s_zTOtzw.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-1HyPjJ-PGJN0jEc3c-uSNw.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-QcjQiIeOAB2npZZxqiiLDQ.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-iC_7baMZcBEdQeLPsUhDNA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-YKDZXQGghsKavBiPwwqfKA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-j3UiDQ2PbJ1a8w1wMG4BxA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-vwKxckwjvAHnZr05I7M1HA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-1AUc3cJWV983MVBGkUx8sw.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-IvaExsFHdfBXPMd-nfsQRw.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-UcZ1DTTj9lmPmRFhe3SKNA.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-JzRg-I1S2dRfBoVZGph3Kg.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-7sWFg96cLdKHHnNWbL5Dew.jar
Apr 18, 2020 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-T0zp9LOgIJXz6lkSmEjuYA.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-i7uZCYzXV-jvg282uuSHIw.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-b2lmK5FM4CxLSROW-qV4-Q.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-8KPA0xPH-3FQWWU0no0M9A.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4r2TFfC07xcxWWFRopnHfw.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4r2TFfC07xcxWWFRopnHfw.jar
Apr 18, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-iCQH_ukwgdhdo4ca-l5WNg.jar
Apr 18, 2020 12:48:30 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4r2TFfC07xcxWWFRopnHfw.jar&uploadType=resumable&upload_id=AEnB2UoZkD9X6Vw3Df1cmuTNpk7uH_ji2YWQ-CWYlD94Qhnqul2oYdpC57r41pJa1JAbOye0MfftEk4_tomSbMxY34yuWKhdHg.
Apr 18, 2020 12:48:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 18, 2020 12:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4r2TFfC07xcxWWFRopnHfw.jar
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 18, 2020 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 18, 2020 12:48:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash lTUGJ0Gi1NeXLx5Iikh2Xw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lTUGJ0Gi1NeXLx5Iikh2Xw.pb
Apr 18, 2020 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 18, 2020 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-18_05_48_36-930948116069486087?project=apache-beam-testing
Apr 18, 2020 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-18_05_48_36-930948116069486087
Apr 18, 2020 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-18_05_48_36-930948116069486087
Apr 18, 2020 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T12:48:36.697Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 18, 2020 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:40.007Z: Checking permissions granted to controller Service Account.
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:43.801Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.517Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.561Z: Expanding GroupByKey operations into optimizable parts.
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.602Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.694Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.733Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.773Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:44.805Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:45.379Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:48:45.467Z: Starting 5 workers in us-central1-c...
Apr 18, 2020 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:49:09.874Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T12:49:14.740Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 18, 2020 12:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:49:37.544Z: Workers have started successfully.
Apr 18, 2020 12:49:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:49:37.578Z: Workers have started successfully.
Apr 18, 2020 12:50:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:50:15.291Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 12:50:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:50:15.491Z: Cleaning up.
Apr 18, 2020 12:50:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:50:15.574Z: Stopping worker pool...
Apr 18, 2020 12:51:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:51:35.766Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 18, 2020 12:51:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T12:51:35.820Z: Worker pool stopped.
Apr 18, 2020 12:51:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-18_05_48_36-930948116069486087 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): fb60743e-1e9d-4a70-bc0f-d83e540339b9 and timestamp: 2020-04-18T12:51:42.589000000Z:
Metric: Value:
read_time 16.718
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 23.417 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 27s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/aj5kdqgsyuxiw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #398
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/398/display/redirect?page=changes>
Changes:
[github] [BEAM-3836] Enable dynamic splitting/checkpointing within the Java SDK
[github] [BEAM-8872] Support split at fraction for OffsetRangeTracker (#11418)
------------------------------------------
[...truncated 283.40 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 18, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 18, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 18, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 18, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-l6r-CBwJS9plsuFRC6HT0A.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-OkjR-ZCK0pJQcs3skyvhSQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-itrD8QAcjXNmH4rrrpAbDQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-fn_KnhRuOv9Ply4tXb0V9Q.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-9KbTIFqjtZ0Yodbwi4dOcw.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-8qGMynq4-TwxLPyGEpYQFA.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Nh6SrsYp-J-j-kcRi36-cQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-XdEP5ThATzzWRyBbOqUexQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-_TDKkC4mI85CZaFQ4nylUA.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-5hImUKzV4u-KyQrXjNBbmg.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-uCHdW0s9BoX7-gGMMOc8yQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-fa4I9MSf5LnRvgWgHsKxzg.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-m2yLdunoqASin635gCWS0A.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-hLH476NLGfWiMmkhZMu6SQ.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-qGMT8dTM7-HjGHUhnV6u6w.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-C84mn97gm9m0GWIKQsDR4w.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-cIbdZMzFnHKmAd8Abcd5ow.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-fbqwSri9Ndxn-VvUzT6VBg.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-7p68alkWCxsEroFMJy36Fg.jar
Apr 18, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-mhdd8RfXa-YRDbyu_uw1BA.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-MT6dOXFyVKX1EdXVE9yLBQ.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tf1XrBUFAoHMNcTPomtpuA.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Ajr43O0CHgC6oNAPsq9lig.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Mwpuo-JtxM3SGAsZPGGuuw.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-OS2AewFH7W8VsV28RL5ncQ.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-7Kl3T_yjoGONRpQEIQq0XQ.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Mwpuo-JtxM3SGAsZPGGuuw.jar
Apr 18, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-9PbGGURk5yGpgwQittNbuQ.jar
Apr 18, 2020 6:45:29 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Mwpuo-JtxM3SGAsZPGGuuw.jar&uploadType=resumable&upload_id=AEnB2Urip40uH8PHs1awy5bauJWtkqjfiq1nA1O-DV9QwxFsAXssZAdBbcgg32OkFg1ZPPHKzp_cDCSDh0rOszQeeOAM-btRoA.
Apr 18, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 18, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Mwpuo-JtxM3SGAsZPGGuuw.jar
Apr 18, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 18, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1JGLMty6edVuZAF7YeEMEw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1JGLMty6edVuZAF7YeEMEw.pb
Apr 18, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-17_23_45_37-2823336900132265084?project=apache-beam-testing
Apr 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-17_23_45_37-2823336900132265084
Apr 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-17_23_45_37-2823336900132265084
Apr 18, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T06:45:37.679Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 18, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:41.096Z: Checking permissions granted to controller Service Account.
Apr 18, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:45.857Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.493Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.536Z: Expanding GroupByKey operations into optimizable parts.
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.560Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.619Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.653Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.689Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:46.715Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:47.142Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:45:47.231Z: Starting 5 workers in us-central1-c...
Apr 18, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T06:45:57.969Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 18, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:46:17.289Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:46:17.315Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 18, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:46:22.752Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 6:46:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:46:42.799Z: Workers have started successfully.
Apr 18, 2020 6:46:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:46:42.839Z: Workers have started successfully.
Apr 18, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:47:20.458Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:47:20.650Z: Cleaning up.
Apr 18, 2020 6:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:47:20.731Z: Stopping worker pool...
Apr 18, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:48:37.228Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 18, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T06:48:37.274Z: Worker pool stopped.
Apr 18, 2020 6:48:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-17_23_45_37-2823336900132265084 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ec49e471-767e-498d-83e4-dd858bd17bb9 and timestamp: 2020-04-18T06:48:43.692000000Z:
Metric: Value:
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
read_time 15.336
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 25.707 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/le3zwlvthc4yw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #397
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/397/display/redirect?page=changes>
Changes:
[APryor] Correct a typo in documentation
[robertwb] [BEAM-9577] Small fixes to portable runner staging.
[robertwb] [BEAM-9768] Gradle command for Python ULR.
[github] [BEAM-8646] Fix external environment on OS X as well. (#11431)
[github] [BEAM-9743] Fix TFRecordCodec to try harder to read/write (#11397)
[github] Updates and clarifications on type safety. (#11449)
------------------------------------------
[...truncated 282.61 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 18, 2020 1:12:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 18, 2020 1:12:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 18, 2020 1:12:14 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Yv348usoS9IRGpKKVrvTEQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-55PTUJ4HUTN15uUXou3wGw.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-DVANYHEv7rfyjhXk-yX6HA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-TDNL_TwOS_8cnPgfs52jDQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-6Sy-odI1V61EZI2Tx-fbXw.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-YkVDPbmFs8RGNYbhl45fvg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-4VUkEYZZWulwUuxan6L9YQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-WH-ZLfcTyE03B99m9bDzuQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-_e9lUy8qWkDnVREvZc4uXg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-iw8kpdxPW5xCkE44iKzRGA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-WLtCgOR0cYz_RhNwtHSkJA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-7Xxjq-qF-7nmv-cWcHSZgA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-so7GSNMaE1I1R_yyXmPezg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-2zuENgTSSpkljZXNvcx7rg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-84Mg3LAk6H80Vx7INUtzOA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lEd4PysNxZodhDVjhKPChg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-8TxEC1WbXefQbQEZYgkeLQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-4Z7l1Ob0c_T9ViUbUQqZxg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-4RLCIv1QpyR-Zkh28db23A.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-4c6R0etUZlYKCVBpW1NvNA.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-O6Fw5xtqayCY3liZoBgxFw.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-DNDUP034RYVX8lfa2Hy_7w.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-h-LU99VuZ9gIMVh53dzkUg.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-R-4OGEq4f58DfPl2Cac16w.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-IsjRt0xgvw0tZZz6SHv8GQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-8CVpyyPItfHwHoK73Rc0LQ.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-O6Fw5xtqayCY3liZoBgxFw.jar
Apr 18, 2020 1:12:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-u3WUm0AGG5CTM3TnRRIL7Q.jar
Apr 18, 2020 1:12:16 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-O6Fw5xtqayCY3liZoBgxFw.jar&uploadType=resumable&upload_id=AEnB2UpjcOapjuXwRDKI_ZjHurCud-OoBTAMUMBpogmBRWeo1hpnSPBMNruRYjZ8HdVnlte9wGA47XZd5ih2t7b5oihn1ytmQw.
Apr 18, 2020 1:12:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 18, 2020 1:12:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-O6Fw5xtqayCY3liZoBgxFw.jar
Apr 18, 2020 1:12:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 18, 2020 1:12:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash oZvp5K8ksozCzSkF0cLdRA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-oZvp5K8ksozCzSkF0cLdRA.pb
Apr 18, 2020 1:12:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 18, 2020 1:12:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-17_18_12_22-787338724424639768?project=apache-beam-testing
Apr 18, 2020 1:12:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-17_18_12_22-787338724424639768
Apr 18, 2020 1:12:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-17_18_12_22-787338724424639768
Apr 18, 2020 1:12:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T01:12:22.103Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 18, 2020 1:12:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:25.773Z: Checking permissions granted to controller Service Account.
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:30.325Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.124Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.158Z: Expanding GroupByKey operations into optimizable parts.
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.190Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.264Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.291Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.319Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.345Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.626Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:31.688Z: Starting 5 workers in us-central1-f...
Apr 18, 2020 1:12:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:58.066Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 1:12:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:12:58.098Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 18, 2020 1:12:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-18T01:12:58.327Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 18, 2020 1:13:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:03.499Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 18, 2020 1:13:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:16.107Z: Workers have started successfully.
Apr 18, 2020 1:13:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:16.138Z: Workers have started successfully.
Apr 18, 2020 1:13:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:51.927Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 18, 2020 1:13:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:52.098Z: Cleaning up.
Apr 18, 2020 1:13:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:13:52.177Z: Stopping worker pool...
Apr 18, 2020 1:16:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:16:08.452Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 18, 2020 1:16:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-18T01:16:08.508Z: Worker pool stopped.
Apr 18, 2020 1:16:14 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-17_18_12_22-787338724424639768 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e3b681e6-6359-44e3-9041-7c02c2c4045e and timestamp: 2020-04-18T01:16:14.321000000Z:
Metric: Value:
read_time 17.563
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 9.29 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7nzgll7wrafki
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #396
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/396/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-9737] Don't use docker create option -u
[github] [BEAM-9674] Don't specify selected fields when fetching BigQuery table
[github] [BEAM-9650] Cleanup documentation on side inputs patterns (#11415)
------------------------------------------
[...truncated 282.89 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 17, 2020 6:51:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-FICMGHOwID-P1QExrB1mbA.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-mQwHRlfuNnQcY6zy0zeTYw.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-auWFinlgvoDpAP4zB_XjtQ.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-u4FSYuIlHegQe1SJUXP3lQ.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-PX0G8S-LgFxgYUYDJ1W51g.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-S-cDCMf5BGZW2prUYTdX-w.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-G5_p2hQMdrpGSphBuE4lsg.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-xf4CHFdYZHzbu3Zm9RzXHw.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-q_xldsvdAf75DlxCQv5gtA.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-TMtKyHyzQ5L8QI42omLZLA.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-PLkriuy4FBDGmQMuA0YAkA.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-kUp64tw4OIjr-bKwr0OVyw.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-zC3Ou93CfBWBHRamR0vQwQ.jar
Apr 17, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-NcrzrHgUvw-eeeMnRw_y7g.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-CTxngnRJx4FeSCs4-Rn9lA.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-e_6lwu12saxc0F8ewA7ndg.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-k_TtjMIjzKxSCyV93a8j2g.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-gCF6D4Msaa82MvLhbsUwjg.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-dDkBGGDYwdam_RtuXEgn_g.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-SdltAibLoyh2mRCp5rmNtw.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-mxA0-0qS6Ti4_NEfOd2-qg.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-SdeCQJCoIZTHOJdIXXt-MA.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-7u3oGZf8xrpFYrNqaX9yCQ.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-3O2-LH5hTHcoy1w4wyisaA.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bI111HJMZip1BmfxuQqrTg.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-Mi9USXdakBU2tM2UEMp7tA.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bI111HJMZip1BmfxuQqrTg.jar
Apr 17, 2020 6:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-3r-vecQYimUDYkGcu2RSig.jar
Apr 17, 2020 6:51:11 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bI111HJMZip1BmfxuQqrTg.jar&uploadType=resumable&upload_id=AEnB2UrB08aYrmcUP7ntVTlJnAA-70TdPTwvu2n51MFxWTXJ0yJ4HeffJsTef7vNEDxq60qGTEGNoCi9be33xjSnG2KITQWAeSU-cK5XVte4CbthpNyxdNE.
Apr 17, 2020 6:51:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 17, 2020 6:51:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-bI111HJMZip1BmfxuQqrTg.jar
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 17, 2020 6:51:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 8IuJm5Iq3Nrjo9qIL3oRVQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8IuJm5Iq3Nrjo9qIL3oRVQ.pb
Apr 17, 2020 6:51:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 17, 2020 6:51:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-17_11_51_17-12742170195902823156?project=apache-beam-testing
Apr 17, 2020 6:51:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-17_11_51_17-12742170195902823156
Apr 17, 2020 6:51:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-17_11_51_17-12742170195902823156
Apr 17, 2020 6:51:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T18:51:17.396Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 17, 2020 6:51:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:21.238Z: Checking permissions granted to controller Service Account.
Apr 17, 2020 6:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:25.764Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.344Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.398Z: Expanding GroupByKey operations into optimizable parts.
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.435Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.550Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.585Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.622Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:26.662Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:27.138Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:27.229Z: Starting 5 workers in us-central1-c...
Apr 17, 2020 6:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T18:51:48.398Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 17, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:54.249Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:54.285Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 17, 2020 6:52:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:59.668Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 6:52:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:51:59.700Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 17, 2020 6:52:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:52:10.470Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 6:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:52:16.368Z: Workers have started successfully.
Apr 17, 2020 6:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:52:16.408Z: Workers have started successfully.
Apr 17, 2020 6:53:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:53:00.010Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 6:53:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:53:00.214Z: Cleaning up.
Apr 17, 2020 6:53:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:53:00.300Z: Stopping worker pool...
Apr 17, 2020 6:54:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:54:29.630Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 17, 2020 6:54:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T18:54:29.679Z: Worker pool stopped.
Apr 17, 2020 6:54:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-17_11_51_17-12742170195902823156 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 965709f7-eeed-4885-abda-b7af96972743 and timestamp: 2020-04-17T18:54:36.989000000Z:
Metric: Value:
fields_read 4375276.0
read_time 23.096
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 36.893 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 39s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/czo5tttpshclu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #395
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/395/display/redirect>
Changes:
------------------------------------------
[...truncated 281.77 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 17, 2020 12:58:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6igVwJU1eawvpQ9UV1rRuA.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-PhvQY9-fncJmBEUegTBY7w.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-W4p9OJJJgoVslF7EiFJOew.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-y974-G491h_44Xdys9_nPA.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-u0D992XEWLtO0XMQQa6lpw.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-veAG3v-xAmEP3bESBKivdQ.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-HdboX0gFFCxHXsrdUcob5g.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT--8qRl9beXL58wBzDaMR_Uw.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-57Qc8CDhTAlRilnilgabLQ.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-zexx9k5Ke6H79wEvia5uBQ.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-dfEzRZ7UhtwDWMzUPrqRaw.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests--ukPDYMKa2JP3cwIsX18vA.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-8SIZmN3oTMafTlZyKdQC1Q.jar
Apr 17, 2020 12:58:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-3pdK2cSrBhkoct6Pe1p0rw.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-QarmjntB5akzauZ0TZXcpQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-xckaKS-O1ZvRXAaAotQhhQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-LWs2nDfs0pBKsdQBTHVajw.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-a7UXB_T40v79rpXKrul6vA.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-vmn171xoUaEslpMsChJbcw.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-7EeEP57Eg2OmI0onnE-3AA.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-YqohwVxmgIBMIw2xC2BBwA.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-to4aoFe8njhdEg0D3CNXkA.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-IayL6PA_3KOyTHnidBIKzQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-kxvpmeMX1beh4zXifT4Okg.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ttA267YNi_Mkvjg5jTtCsQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ttA267YNi_Mkvjg5jTtCsQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-HdeM5uAShF4R6hrlnJQqgQ.jar
Apr 17, 2020 12:58:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-BXz-AQ9bZ0YlVY5rCxn6kw.jar
Apr 17, 2020 12:58:38 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ttA267YNi_Mkvjg5jTtCsQ.jar&uploadType=resumable&upload_id=AEnB2UqpbN3CuNpKeX-B-EoS0S4VdLyeJIpczDc0-1Xu3MLzCJTXGItf_oenaWCy28taRGBi_vvecUnQyHyxu1oy2ckA3OrW5g.
Apr 17, 2020 12:58:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 17, 2020 12:58:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ttA267YNi_Mkvjg5jTtCsQ.jar
Apr 17, 2020 12:58:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 6 seconds
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash NaJBu_Iz_qs6s4OUNMO2MQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-NaJBu_Iz_qs6s4OUNMO2MQ.pb
Apr 17, 2020 12:58:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 17, 2020 12:58:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-17_05_58_44-1642187335101809326?project=apache-beam-testing
Apr 17, 2020 12:58:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-17_05_58_44-1642187335101809326
Apr 17, 2020 12:58:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-17_05_58_44-1642187335101809326
Apr 17, 2020 12:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T12:58:44.043Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 17, 2020 12:58:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:48.141Z: Checking permissions granted to controller Service Account.
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:51.941Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.586Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.623Z: Expanding GroupByKey operations into optimizable parts.
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.695Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.788Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.823Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.859Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 17, 2020 12:58:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:52.897Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 17, 2020 12:58:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:53.317Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 12:58:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:58:53.408Z: Starting 5 workers in us-central1-f...
Apr 17, 2020 12:59:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T12:59:16.183Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 17, 2020 12:59:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:59:20.276Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 12:59:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:59:40.025Z: Workers have started successfully.
Apr 17, 2020 12:59:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T12:59:40.054Z: Workers have started successfully.
Apr 17, 2020 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T13:00:15.826Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T13:00:16.025Z: Cleaning up.
Apr 17, 2020 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T13:00:16.113Z: Stopping worker pool...
Apr 17, 2020 1:02:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T13:02:04.576Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 17, 2020 1:02:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T13:02:04.634Z: Worker pool stopped.
Apr 17, 2020 1:02:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-17_05_58_44-1642187335101809326 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c8a690ae-2ccb-40a9-b5ec-072f7c2606eb and timestamp: 2020-04-17T13:02:11.690000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.583
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 45.243 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 43s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/xojoxchb4r6i4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #394
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/394/display/redirect>
Changes:
------------------------------------------
[...truncated 283.18 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 17, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 17, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 17, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 17, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-3KE12C3r-RMHHfZmN0Nhog.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-tXgfXOwPrbGzWNtGVmjOYg.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-6-RBw0qoTQtdM4sOaogC4A.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-LetERWndDbNAE03cVxJOLg.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-WJYuKIXY15wvF1jPWuG16w.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-PmZu4auci4W3lmhoefXRvQ.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-BpS4lWf-1LUBBVbrM4a7Uw.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-aJgKoOgLA8HaJSmmbu8Vqw.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-yV9r5l2z5gnrw78pEVNqZg.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-J5Ui1zLhmT649OBm1XXUeg.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-OeHTmH6gAn7CuVR9QJgMfw.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-RZEMCmTvL33iPdxLokJI_w.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-FowMauYaeDla3bGcwh-P0g.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-1s4gnzzhucNUd07wHNaz2Q.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-kBCpfdmhKWX9jckRocBZdw.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-DNIJ90eV1inrQ4NyIZJ8gQ.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-rdDDwe9pKbV--2-BjTAp7A.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-gC4Oa0G9TtoRQlVLtqqN5g.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-2S5i0IfOdn4GZWcsXCrPlw.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-cNsbPwdPLml3bCWtBUuhsg.jar
Apr 17, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-LG36BRk7ciTNvuksQP9CtQ.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-aPSDrXF0b_PyywSSq8gh6Q.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Oh4mUxYkW2mBEQrMqES1nA.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-pd8D3CdRcG6ARBoZ6mh9VA.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-yqQh-EA19QWHeVbLQa-pKQ.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-1-XZulfCpprB6ezEU74yRg.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ybo2jWln37YGTbjF_F5gHQ.jar
Apr 17, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ybo2jWln37YGTbjF_F5gHQ.jar
Apr 17, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ybo2jWln37YGTbjF_F5gHQ.jar&uploadType=resumable&upload_id=AEnB2Uo89o_OYYPoDlLq0cieCtuisveYfHiZxIXqhXVCH4mrolmz0WpF0R6MMWj9W2J1Y_4jEkKCHZej_ljPdNojo7jxR_ntSA.
Apr 17, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 17, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Ybo2jWln37YGTbjF_F5gHQ.jar
Apr 17, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 17, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash LhMrxmhJioiErUmRzc_QZA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LhMrxmhJioiErUmRzc_QZA.pb
Apr 17, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 17, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-16_23_45_41-6893042653972751203?project=apache-beam-testing
Apr 17, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-16_23_45_41-6893042653972751203
Apr 17, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-16_23_45_41-6893042653972751203
Apr 17, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T06:45:41.075Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 17, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:44.480Z: Checking permissions granted to controller Service Account.
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:47.792Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.425Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.467Z: Expanding GroupByKey operations into optimizable parts.
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.500Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.571Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.605Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.632Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.663Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:48.995Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:45:49.071Z: Starting 5 workers in us-central1-c...
Apr 17, 2020 6:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T06:46:02.675Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 17, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:46:14.663Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 6:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:46:14.696Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 17, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:46:20.144Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:46:32.830Z: Workers have started successfully.
Apr 17, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:46:32.859Z: Workers have started successfully.
Apr 17, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:47:13.451Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:47:13.668Z: Cleaning up.
Apr 17, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:47:13.758Z: Stopping worker pool...
Apr 17, 2020 6:48:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:48:35.524Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 17, 2020 6:48:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T06:48:35.568Z: Worker pool stopped.
Apr 17, 2020 6:48:41 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-16_23_45_41-6893042653972751203 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b39857e4-abc2-4b97-8cb9-ecb4af787152 and timestamp: 2020-04-17T06:48:41.871000000Z:
Metric: Value:
read_time 21.338
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 21.64 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 24s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7iwhkw6yvwrd4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #393
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/393/display/redirect?page=changes>
Changes:
[huangry] Include user distritribution into metric-dedicated validate runner test.
[huangry] update inline comment.
[huangry] To allow only assert on committed metrics, bypassing the attempted ones.
[huangry] Use distribution matcher for testing.
[pabloem] [BEAM-9769] Ensuring JSON is the default export format for BQ sink
[github] Merge pull request #11436: Document Beam Schemas (forked PR)
[github] [BEAM-9648]: DirectRunner should return null on timeout (#11275)
[github] [BEAM-5379] Ignore go.sum files in RAT checks
[github] [BEAM-5605, BEAM-2939] Add support for FnApiDoFnRunner to handle split
------------------------------------------
[...truncated 282.34 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 17, 2020 1:07:51 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 17, 2020 1:07:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 17, 2020 1:07:52 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-1jloBROLB1zVqAZ35QXjww.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-VQMRM83oF3mFyMAYd4tnmw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-fO04ni4M9P8Ep-qlSqSjRg.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-oqyMuGFt0wFPiR3o1D_JAw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-tZH93pHG7j37T-pTwrPAxQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tJVFJlTg3A201iEP_VtgrQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-aZ2C0sc9ESXGWhG80TqS2g.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-fblZi0sn56wnmhwPgu0tVw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-3an51pXQy-pDq7CtVZqPwA.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-vocWnM8bL5sqXYx2AukNYQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-_yI8XucvTItRPC8vcjOoZQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-6uiz7P1PyeMfH4ib23_hpA.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-6mUEE2lGIOtX6NTAZ2t4tQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-4C67gjpNcnra2TjXKgbvZQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-mcBiDTkXszc57CZhWU9lsg.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-lkW-k5MYTdhH4Bw37jrjiQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MHxQ5mwHPIq5oyOA0Ogl9g.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-IVqD4UszNdgwKFcMKXvuJw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-CcHPQREtSQ2yDARVuWLkUQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Dmffa22Y3WPQtoaCix7RsA.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-ZkM8fD65KgNLWXzpxI-mLA.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-XtW24PxnXLUnRB7LQG9R1Q.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-w-4E4vIZiRIo9BtuazHlow.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-tH-TTEosB7Vh7Gnaiea6zw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cEZ0S1ptETqUkwmODWgUvA.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-ndFzD4hJQnTx8EB8RBy8Rw.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tdOO_nejCAf0CcJOjMi7YQ.jar
Apr 17, 2020 1:07:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cEZ0S1ptETqUkwmODWgUvA.jar
Apr 17, 2020 1:07:55 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cEZ0S1ptETqUkwmODWgUvA.jar&uploadType=resumable&upload_id=AEnB2UqEzqHLqC6c4Pqn-eAUJc8aTESNKXRfsU0WXp78pLlSFT0TrJy0BgolxQKHWilQQNWHLi2yxBbST2Wj9yDxzDxi1p1avQ.
Apr 17, 2020 1:07:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 17, 2020 1:07:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cEZ0S1ptETqUkwmODWgUvA.jar
Apr 17, 2020 1:08:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 8 seconds
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash a5Xe6Em69jpuD7lBIMbdEQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-a5Xe6Em69jpuD7lBIMbdEQ.pb
Apr 17, 2020 1:08:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 17, 2020 1:08:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-16_18_08_01-8486023506646537306?project=apache-beam-testing
Apr 17, 2020 1:08:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-16_18_08_01-8486023506646537306
Apr 17, 2020 1:08:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-16_18_08_01-8486023506646537306
Apr 17, 2020 1:08:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T01:08:01.927Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 17, 2020 1:08:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:05.382Z: Checking permissions granted to controller Service Account.
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:09.569Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.138Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.182Z: Expanding GroupByKey operations into optimizable parts.
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.220Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.310Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.336Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.368Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.402Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.774Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 1:08:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:10.858Z: Starting 5 workers in us-central1-c...
Apr 17, 2020 1:08:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-17T01:08:30.652Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 17, 2020 1:08:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:08:41.297Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 17, 2020 1:09:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:09:01.150Z: Workers have started successfully.
Apr 17, 2020 1:09:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:09:01.182Z: Workers have started successfully.
Apr 17, 2020 1:09:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:09:43.613Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 17, 2020 1:09:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:09:43.824Z: Cleaning up.
Apr 17, 2020 1:09:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:09:43.908Z: Stopping worker pool...
Apr 17, 2020 1:11:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:10:59.869Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 17, 2020 1:11:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-17T01:10:59.908Z: Worker pool stopped.
Apr 17, 2020 1:11:08 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-16_18_08_01-8486023506646537306 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f8cf4e7f-ae54-4c3a-aeac-ffceefac6fdf and timestamp: 2020-04-17T01:11:08.893000000Z:
Metric: Value:
read_time 22.584
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 25.834 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 31s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/u2hyvewchz2ys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #392
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/392/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9765] Exclude module-info.class from vendored Calcite.
[github] [BEAM-9678] Create Go SDK introduction kata (#11340)
------------------------------------------
[...truncated 284.43 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 16, 2020 6:48:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-ONOvaTXQbWOw5rxZCjJ3xA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-TGtuC-zT1mTX_-sh5BH1nA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-uIAN_Me2szKqHHIO-C88xQ.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-C7ltaDcDnKrkckkqMFhdwA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-jaMRL06mzFxl428M57Pv3w.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-6u1lHdoUPk0WkNQxpsFeBQ.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Z4rT9TXSSAwa1q-p84l0xA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-9xHDFTDl-T_5uk5mMLzIGw.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-8UFbavXmaB8JFkCSneDaEg.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-navw60Nr4tR6fbrgeR0qNQ.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-6Z6TSnJbSochxnPUgpHlzg.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-ybdkqq0lVFEaloXopncyTw.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-s6hWW1LYJ3xtK_SpV5_rHw.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-AWfkHafVbJTEu15-ebB8DA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-x7R3KK9wzwcmbQJni0qkmw.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-mFpF7BDS_e5KN1xqLxobLg.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-oa0a6kBOvhH6L1adWIzhSA.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-aFl3Z1s-FqzmzPKxDDAN1Q.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-2QyHt9izZdY3DZt8zu_OvQ.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-QPwY-CTQt_A0ijtCoybPUw.jar
Apr 16, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-rHYpyBIjAywaH4vNQPi7Bg.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-04HxnvjCAwOCj5x9KnL5vA.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-VPR6EDNHCAISHThStXuWyQ.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-jOkETn_GEW5hUPOcxbmgmw.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-GoeEtzGr3eB-j3VI54uPLw.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YaMwL46HusRXV-JAM6bREQ.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-5WpxOrnCBN2-SyghY47EvQ.jar
Apr 16, 2020 6:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YaMwL46HusRXV-JAM6bREQ.jar
Apr 16, 2020 6:48:59 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YaMwL46HusRXV-JAM6bREQ.jar&uploadType=resumable&upload_id=AEnB2Uokx_hHjr831gPx2toRQwbh4NsVR3f2R60MZxDniht18-Ov-VKko1Mu4CxsPwk_zakE4XwwOSJQrYEo7uHvJKGWQuh4vw.
Apr 16, 2020 6:48:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 16, 2020 6:49:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-YaMwL46HusRXV-JAM6bREQ.jar
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 9 seconds
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 16, 2020 6:49:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 16, 2020 6:49:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash kX37slq8QbD8ZvVo9rFiSw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kX37slq8QbD8ZvVo9rFiSw.pb
Apr 16, 2020 6:49:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 16, 2020 6:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-16_11_49_07-13685756303418674198?project=apache-beam-testing
Apr 16, 2020 6:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-16_11_49_07-13685756303418674198
Apr 16, 2020 6:49:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-16_11_49_07-13685756303418674198
Apr 16, 2020 6:49:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T18:49:07.687Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 16, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:10.989Z: Checking permissions granted to controller Service Account.
Apr 16, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:14.811Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 16, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.036Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 16, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.069Z: Expanding GroupByKey operations into optimizable parts.
Apr 16, 2020 6:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.093Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.153Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.176Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.198Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.218Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.497Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 6:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:16.557Z: Starting 5 workers in us-central1-f...
Apr 16, 2020 6:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T18:49:29.588Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 16, 2020 6:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:40.205Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 6:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:40.241Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 16, 2020 6:49:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:49:45.704Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 6:50:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:50:01.131Z: Workers have started successfully.
Apr 16, 2020 6:50:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:50:01.163Z: Workers have started successfully.
Apr 16, 2020 6:50:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:50:34.706Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 6:50:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:50:34.955Z: Cleaning up.
Apr 16, 2020 6:50:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:50:35.020Z: Stopping worker pool...
Apr 16, 2020 6:52:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:52:56.594Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 16, 2020 6:52:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T18:52:56.627Z: Worker pool stopped.
Apr 16, 2020 6:53:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-16_11_49_07-13685756303418674198 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f26d198e-9b1d-4c2b-8b72-10aa196193ec and timestamp: 2020-04-16T18:53:02.274000000Z:
Metric: Value:
read_time 17.254
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
fields_read 4375276.0
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 4 mins 15.13 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 22s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/22l2pezjcuthk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #391
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/391/display/redirect?page=changes>
Changes:
[kamil.wasilewski] [BEAM-8671] Migrate Load Tests to Python 3.7
[kamil.wasilewski] Remove outdated doc for ReadFromBigQuery transform
[jkw] Fix typo
[echauchot] Add GBK load tests script for spark structured streaming runner
[echauchot] Add CoGBK load tests script for spark structured streaming runner
[echauchot] Add Combine load tests script for spark structured streaming runner
[echauchot] Add Pardo load tests script for spark structured streaming runner
------------------------------------------
[...truncated 282.99 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 16, 2020 12:56:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 12:56:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 16, 2020 12:56:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 12:56:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 16, 2020 12:56:04 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 16, 2020 12:56:04 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 16, 2020 12:56:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 16, 2020 12:56:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-YI3E5-iZ6lUwvd5MqdQmgQ.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LCQOfoZgdti4ywHRaMG3-w.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-8UbqfFoEzsP6v364VzxBxw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ASXM33-K8A_OVpUU9D_Mxg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-m0s2o0aXU8x60btd9IY3Sw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-IoUwQGsLpFZtkiV2v8WIcw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-7VYXd7NfIsQJ75TFKHj7CQ.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-dWahq2RbGD_gKBJdDH_4Zw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-1HDmx6b8LdVzQD24EuLYfw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-kXBCWss-h3McHep_TH7oMg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-E8Hny2aTBjoFIdZlUtqx6w.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-zDDE_1ZR83v7M05Q6oQZUg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-MDqUbMgZMa4y40yqQwAo5A.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-aU5hkL-BD1CrXAy1Ji6sPA.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-QjAPssK8lGL1XRHHoOaWdg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-8SKGlqX9iSQMjqMtzA5OMA.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-W6br5ndCuaxjJ8-WvBqcbw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-hpCuVQfdnRuin5R0AAfzpw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-oHhritN7smQg0xqqLqwv7w.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-ILp_AKNwyEyGyFglOF7zHg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-fyfyYpHAZVgVk5BJUcg-_A.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-fjx8bjbo_zBWMsscTcE9Uw.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-OJuPsE9GZpPqLkA7rEFNbA.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-VgALq3_zXlX_w34UufZZzg.jar
Apr 16, 2020 12:56:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-Y3a6jHtHZR2Qu791WIA9Ng.jar
Apr 16, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-G49JPedUyxy_77PDg61lEA.jar
Apr 16, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-BCgMxuupiER_QyNw0PoicA.jar
Apr 16, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-G49JPedUyxy_77PDg61lEA.jar
Apr 16, 2020 12:56:07 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-G49JPedUyxy_77PDg61lEA.jar&uploadType=resumable&upload_id=AEnB2UqbMoAPD_eKcls1jCA325T1x9maTG-F6jzRtEG2NYT3089L-Yp-WvGeHrX0iSYM0TV9PWOoib17sZMtc2NPiYk-JimxoA.
Apr 16, 2020 12:56:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 16, 2020 12:56:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-G49JPedUyxy_77PDg61lEA.jar
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 16, 2020 12:56:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash gE7iz-rLa_RjtapEh0JYLA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gE7iz-rLa_RjtapEh0JYLA.pb
Apr 16, 2020 12:56:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 16, 2020 12:56:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-16_05_56_13-5801989627753206820?project=apache-beam-testing
Apr 16, 2020 12:56:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-16_05_56_13-5801989627753206820
Apr 16, 2020 12:56:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-16_05_56_13-5801989627753206820
Apr 16, 2020 12:56:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T12:56:13.418Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 16, 2020 12:56:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:17.360Z: Checking permissions granted to controller Service Account.
Apr 16, 2020 12:56:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:21.575Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.336Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.383Z: Expanding GroupByKey operations into optimizable parts.
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.433Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.527Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.572Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.627Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:22.689Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:23.083Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 12:56:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:23.166Z: Starting 5 workers in us-central1-f...
Apr 16, 2020 12:56:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T12:56:31.852Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 16, 2020 12:56:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:50.103Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 12:56:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:50.142Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 16, 2020 12:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:56:55.577Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 12:57:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:57:09.227Z: Workers have started successfully.
Apr 16, 2020 12:57:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:57:09.272Z: Workers have started successfully.
Apr 16, 2020 12:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:57:44.584Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 12:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:57:44.830Z: Cleaning up.
Apr 16, 2020 12:57:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:57:44.934Z: Stopping worker pool...
Apr 16, 2020 12:59:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:59:38.330Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 16, 2020 12:59:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T12:59:38.384Z: Worker pool stopped.
Apr 16, 2020 12:59:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-16_05_56_13-5801989627753206820 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 32820884-9cc4-44de-842a-e3745de888c7 and timestamp: 2020-04-16T12:59:44.541000000Z:
Metric: Value:
read_time 19.538
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.155 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 49.101 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 47s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/kdpptfnejh5tg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #390
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/390/display/redirect?page=changes>
Changes:
[github] Merge pull request #11151 from [BEAM-9468] Hl7v2 io
------------------------------------------
[...truncated 283.39 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 16, 2020 6:45:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-wxV3QDQJC0hch1p_WcyhOA.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-7ExFk6I81cBVleNlGDNCSA.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-dUABdg2x3YHHoIoYOxmlHg.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-DqjfGRY_UZoCSMemqT2_sQ.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-XNyIJAIVrF3l6Wfc8H2WUA.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ic1otSFl79Ce0yS4VM3_3w.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests--7jmtYCbSKOB1dMs9bZ0Sg.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-O56GQtduEMq31-tqUZeUUw.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-XSwU838hHDiQ4BqxX2V9hA.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-rZtboR5zOAFADMwEpEXeMQ.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-yuMtA94i1-HzIo9o9qH77A.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-jsq_3MDOIiSkiYJZvrRkvw.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-3uZc7A1QqfuycYJJcKVFZQ.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-fkQa03OfDIFDRZHtsW9O9Q.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-FC5rsknGBMAc2upI4IYtYA.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-X-P1x_0yqxlCH0nHGB6OJQ.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Qys94JHaPh5AQrnSb2Mytw.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-9SMB9CyufdZLBSjb-K3m4w.jar
Apr 16, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-KbBuiWua8Us8uLQswI4rNQ.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-kT3NOkYLebgnR1XYL-HQng.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-PfMDyiHK0fwBRirWUR0xXA.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Xgk1H8GfZMzGVY-4glovlg.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-vclJM6c1C3nrH3dfL0s6Ew.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-tcfrmPOgJ3qacb4CDgEKag.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1guzjDkboERPofAqwgDlDA.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-eEGI_VwnHYpOoNmJi2x2aQ.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gU_kOH8Bk5URkygQB9hD_g.jar
Apr 16, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1guzjDkboERPofAqwgDlDA.jar
Apr 16, 2020 6:45:36 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1guzjDkboERPofAqwgDlDA.jar&uploadType=resumable&upload_id=AEnB2Upfg5QOCB6vm2HZFH1m_z4Pp15AugTapmBzET-87LzWdbVYRhzqdzwo2nKyr793WAYSJSX-FMwt8VEjbM0ygAqK-5gYCw.
Apr 16, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 16, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-1guzjDkboERPofAqwgDlDA.jar
Apr 16, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 28 files newly uploaded in 7 seconds
Apr 16, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 16, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash rAlR-NfVS-0e5m8J-Q6eAA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rAlR-NfVS-0e5m8J-Q6eAA.pb
Apr 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 16, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-15_23_45_42-9194650758086045495?project=apache-beam-testing
Apr 16, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-15_23_45_42-9194650758086045495
Apr 16, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-15_23_45_42-9194650758086045495
Apr 16, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T06:45:42.770Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 16, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:46.389Z: Checking permissions granted to controller Service Account.
Apr 16, 2020 6:45:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:50.805Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.348Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.400Z: Expanding GroupByKey operations into optimizable parts.
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.438Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.545Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.585Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.622Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:51.656Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:52.171Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:45:52.248Z: Starting 5 workers in us-central1-f...
Apr 16, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T06:46:16.651Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:46:19.635Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:46:19.671Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 16, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:46:25.111Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:46:39.506Z: Workers have started successfully.
Apr 16, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:46:39.565Z: Workers have started successfully.
Apr 16, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:47:13.752Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:47:13.981Z: Cleaning up.
Apr 16, 2020 6:47:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:47:14.078Z: Stopping worker pool...
Apr 16, 2020 6:49:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:49:12.323Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 16, 2020 6:49:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T06:49:12.378Z: Worker pool stopped.
Apr 16, 2020 6:49:18 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-15_23_45_42-9194650758086045495 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 11f69879-6a67-4260-a110-a79c3030405c and timestamp: 2020-04-16T06:49:18.118000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.013
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 53.626 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 0s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/x7unxcgk6p6rq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #389
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/389/display/redirect?page=changes>
Changes:
[amaliujia] add 2.20.0 blog post
[amaliujia] add a known issue
[thw] Fix py37-lint
[thw] Maven compatible publish repository authentication via settings.xml
[amaliujia] fixup! update 2.20.0 date
[github] [BEAM-9729, BEAM-8486] Runner-side bundle registration cleanup. (#11358)
[github] Add new release 2.20.0 to beam website (#11285)
------------------------------------------
[...truncated 286.00 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 16, 2020 12:54:58 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 16, 2020 12:54:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 16, 2020 12:54:59 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-koBx1QbkZhfyq5IVcAlUWw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-w-yKMFS6a5bBcmiSR6epsA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-l8GBvhql-Xansr5J7ZQjwg.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-4rceEbu3tKKba08as8ZrrA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-aQ0zYlb-HTfui3uZaXuFmw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-A0Rj2foOX2fCsVQbNjEeEQ.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-wm5clhkqjwlrHTaHHyxjQA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-6KdmZpvSEkKi7sSZFD6yNA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-f449j0lyBExkx95ZNyMLYg.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-NhOLPSenayQ3LGmWpWM5tQ.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-88P35QwRBDlDpE7zxUsJSQ.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-mkoAScErm7x1qbAoCWlwaw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-ZtrD_oKUpqoxTXhGXeeBgA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-p2mZU-4akR_aI4rwFsHgPg.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-3JHCLbnbLi29d7NjrK4YFA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Vf8EkklPFpw3iUlituGY0w.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-7gDkMWJRJr_qP6AcvVR32Q.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-hugiRNYW9ybM7Z_A9xGIow.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-FBrUVHs5_zm5lhlUX1PyzQ.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-BfwQCYdgpUBPAppuNOQ61Q.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-uvgEWCQV4j7UmkjuyGr7sA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-IDApxm0MxYlBizatCv7lCg.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ccoJgGHU5CgUE2OkGeHDCw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-GZ5or4GZzws0IW6yOmaIIA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ccoJgGHU5CgUE2OkGeHDCw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-fLJGLaJj1GqUd_K_CZ11rA.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-mmCf6y7NPYs2hkoPtNiVRw.jar
Apr 16, 2020 12:55:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-XbKlfQnWK-ir1r7Po7_SJA.jar
Apr 16, 2020 12:55:01 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ccoJgGHU5CgUE2OkGeHDCw.jar&uploadType=resumable&upload_id=AEnB2UqwJskdcZfALHzWeAl26a5LkvBVY6wpJjT_cs8MJAuh2sygzd7RYq6PAP3fUtldNE3JuS3Xp_Sz-RN2UT8fVfnmwvF1Bg.
Apr 16, 2020 12:55:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 16, 2020 12:55:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ccoJgGHU5CgUE2OkGeHDCw.jar
Apr 16, 2020 12:55:05 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 16, 2020 12:55:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash q7l8nWNfabRK3YAE-dZWmA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-q7l8nWNfabRK3YAE-dZWmA.pb
Apr 16, 2020 12:55:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 16, 2020 12:55:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-15_17_55_06-6971256062005207406?project=apache-beam-testing
Apr 16, 2020 12:55:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-15_17_55_06-6971256062005207406
Apr 16, 2020 12:55:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-15_17_55_06-6971256062005207406
Apr 16, 2020 12:55:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T00:55:06.931Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 16, 2020 12:55:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:10.420Z: Checking permissions granted to controller Service Account.
Apr 16, 2020 12:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:14.722Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 16, 2020 12:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.590Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 16, 2020 12:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.630Z: Expanding GroupByKey operations into optimizable parts.
Apr 16, 2020 12:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.668Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.756Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.780Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.818Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:15.856Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:16.318Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 12:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:16.398Z: Starting 5 workers in us-central1-c...
Apr 16, 2020 12:55:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-16T00:55:35.370Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 16, 2020 12:55:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:55:45.161Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 16, 2020 12:56:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:56:09.106Z: Workers have started successfully.
Apr 16, 2020 12:56:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:56:09.159Z: Workers have started successfully.
Apr 16, 2020 12:56:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:56:49.165Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 16, 2020 12:56:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:56:49.354Z: Cleaning up.
Apr 16, 2020 12:56:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:56:49.435Z: Stopping worker pool...
Apr 16, 2020 12:58:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:58:09.642Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 16, 2020 12:58:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-16T00:58:09.681Z: Worker pool stopped.
Apr 16, 2020 12:58:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-15_17_55_06-6971256062005207406 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 09c67e9b-5519-46d6-b5a4-c33a88f11f50 and timestamp: 2020-04-16T00:58:15.870000000Z:
Metric: Value:
read_time 18.36
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 26.579 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/h4s32huohxbkq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #388
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/388/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-8466] Make strip_iterable more strict
[lcwik] [BEAM-2939] Update unbounded source as SDF wrapper to resume
[pabloem] Fixing type names for BQ Avro Tools
[lcwik] fixup! Fix spotbugs warning
[kcweaver] [BEAM-9756] Nexmark: only use --region in Dataflow.
[lcwik] [BEAM-9577] Fix test to create urls from paths which are compatible with
[pabloem] Removing underscore from _ReadFromBigQuery to make it external. It
[ehudm] [BEAM-9119] Disable flaky test
[github] [BEAM-8889] add gRPC suport in GCS connector (behind an
------------------------------------------
[...truncated 283.17 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 15, 2020 7:20:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 7:20:54 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 15, 2020 7:20:54 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 7:20:55 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 15, 2020 7:20:55 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 15, 2020 7:20:55 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 15, 2020 7:20:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 15, 2020 7:20:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-ygkxvW9zsGblqjTkjFMC_Q.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Geio5dR-epN7QprSZqOITQ.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-sQ_ApiTd9Gvh4UqG3_8rSw.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-YqsEU37jYWV6QnMHZRXtOw.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-hm3SkhZTaQF-xgzcs2R70w.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-WIOC9nebXgY527u3elzkzQ.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ciTO6mDV3m4oKMc9Gpp4cg.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-i-_3YxBqrQJtVFclAraKfw.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xYlbHWNBZftKa9yJkWGmfQ.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-eskMmJmRWZ8jrvVbc10vgg.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-Fcgb6oN2yL3V8dXoikdrcg.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-71zXgxYVNvymAbxZKF6yVQ.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-NeFxPgCTM2BF8LmiNPozCg.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-SuerWRTyKtDmh90hz1q4Sw.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-ypNbzLi_1mRPpxMiVBABwg.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-OTAMmqbaxMsdb6PgrdxW_g.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-tsLoV6F3Dfm2MnmLGzomDA.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT--cbfmVq2HKxvlNKDv6A-aA.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-SbCHbIBxByoygKrfcIPL7w.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-Bwx8SM2S4uXsnAjmyRCUEA.jar
Apr 15, 2020 7:20:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-YhjISZl_xi7P7WtXsrHIVg.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3vNZrpBQD2Ju8Aur9HRW1Q.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-1Hh_S2fk8io_O2IVXKrwWQ.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-W3qY45St3CQDMu8RZZbgMg.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-41753N_tGNlbts6MRPK-bQ.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-b5VKMGDhPHJFsrXBKSdAqw.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3vNZrpBQD2Ju8Aur9HRW1Q.jar
Apr 15, 2020 7:20:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-YUbT630gpdeskt8xqmyFiQ.jar
Apr 15, 2020 7:20:58 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3vNZrpBQD2Ju8Aur9HRW1Q.jar&uploadType=resumable&upload_id=AEnB2UrP569K9C1qbNqBlEAxOG4AV8X9kdMxhnKANaxD30qsrWxxvSDoxUUMAdF-GFi2oJEDhCVZMLwTqAoTqjhXvlmd3tUf3A.
Apr 15, 2020 7:20:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 15, 2020 7:21:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3vNZrpBQD2Ju8Aur9HRW1Q.jar
Apr 15, 2020 7:21:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 9 seconds
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash soWPKHm-onw9qARxqIimzg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-soWPKHm-onw9qARxqIimzg.pb
Apr 15, 2020 7:21:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 15, 2020 7:21:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-15_12_21_06-4687633547818021223?project=apache-beam-testing
Apr 15, 2020 7:21:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-15_12_21_06-4687633547818021223
Apr 15, 2020 7:21:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-15_12_21_06-4687633547818021223
Apr 15, 2020 7:21:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T19:21:06.999Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 15, 2020 7:21:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:11.623Z: Checking permissions granted to controller Service Account.
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:15.294Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.108Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.137Z: Expanding GroupByKey operations into optimizable parts.
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.163Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.243Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.275Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.303Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.346Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:16.986Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 7:21:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:17.070Z: Starting 5 workers in us-central1-c...
Apr 15, 2020 7:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T19:21:32.577Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 15, 2020 7:21:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:21:47.179Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 7:22:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:22:05.648Z: Workers have started successfully.
Apr 15, 2020 7:22:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:22:05.689Z: Workers have started successfully.
Apr 15, 2020 7:22:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:22:46.059Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 7:22:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:22:46.297Z: Cleaning up.
Apr 15, 2020 7:22:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:22:46.393Z: Stopping worker pool...
Apr 15, 2020 7:24:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:23:59.209Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 15, 2020 7:24:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T19:23:59.259Z: Worker pool stopped.
Apr 15, 2020 7:24:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-15_12_21_06-4687633547818021223 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 975170b5-4d21-4558-b8e3-11488aad7313 and timestamp: 2020-04-15T19:24:04.567000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.584
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 18.02 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 23s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/jsl4prelepql2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #387
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/387/display/redirect>
Changes:
------------------------------------------
[...truncated 284.48 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 15, 2020 12:47:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 15, 2020 12:47:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 15, 2020 12:47:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-cJUXPiUPvsAKTIHnmNMQsA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-3OTGSAOoikhadt8p2YKmKA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-zFpmd9PoBEb37FGPMZHpkA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-X6F1YLFeUX9J7Yvgg9d7AQ.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-QZXTKVWMNgQlKmfh6Pm8xg.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-iicY4Ki8F3KEG2fz1trgGA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-pLQ6NDCBSnn6ZmoX7g0RQA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-x0lgC0k2SEvnEq3qaLbkRQ.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-HVu6e43WdMGl2eI7mU340w.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-LsaBlWhGj1eMuGHfc2bebw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-rADTBPfkVAsP1j9LcLhBQw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-JP-vgHeTFMK8L-ISnM0Myg.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-vToLRO5yGmegxZpfFhA-oQ.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-GMVm-bkMAz1KtkYI-lV78w.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Ofzmx7J8XwmvxHJoLIwExw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-9AygBabhW1p8UpXrn1n4lw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-XRsH9dkvDU942NtFLXVL9Q.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-sxaC7HPM4f-PjGUOLgw2dA.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-_mjZ5r7gCSMTIRzk4jHhMg.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-WmbYbxDydnA8LKuXLPzpNg.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-8Jht46SyWjmZqe8AYPLEcQ.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-FTJFMTzG1IrUsTy5wInaXw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-pIrqUy229eiNGCRNCWxH-A.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-VCxba8flEjW-GMCTz1sD7Q.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-qfnlKgDJBPhX6JCvfsGZVg.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-5gZNGDKd2HCM5Hm5ezuqbw.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--rdv9mKCoK5_R1zZTxz7xQ.jar
Apr 15, 2020 12:47:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--rdv9mKCoK5_R1zZTxz7xQ.jar
Apr 15, 2020 12:47:10 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--rdv9mKCoK5_R1zZTxz7xQ.jar&uploadType=resumable&upload_id=AEnB2UomtxeVBvVcCW4Ouc9qXs20KkxCsTLztxweluK1fn2vPwhIDut8jcPi63CnGrEhQsYHIalL1sl_tygdQwFKmrIjhBd2POVjg23tgd5wUseaUROHhVQ.
Apr 15, 2020 12:47:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 15, 2020 12:47:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT--rdv9mKCoK5_R1zZTxz7xQ.jar
Apr 15, 2020 12:47:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 11 seconds
Apr 15, 2020 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 15, 2020 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 15, 2020 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 15, 2020 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 15, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 15, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash yTBzbmCwRBtdJ-q6uwH-cA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yTBzbmCwRBtdJ-q6uwH-cA.pb
Apr 15, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 15, 2020 12:47:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-15_05_47_19-5946839451280560205?project=apache-beam-testing
Apr 15, 2020 12:47:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-15_05_47_19-5946839451280560205
Apr 15, 2020 12:47:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-15_05_47_19-5946839451280560205
Apr 15, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T12:47:19.691Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 15, 2020 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:22.696Z: Checking permissions granted to controller Service Account.
Apr 15, 2020 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:27.448Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 15, 2020 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.252Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 15, 2020 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.294Z: Expanding GroupByKey operations into optimizable parts.
Apr 15, 2020 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.327Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.409Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.448Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.489Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.526Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.880Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:28.960Z: Starting 5 workers in us-central1-f...
Apr 15, 2020 12:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:55.876Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 12:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:47:55.902Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 15, 2020 12:47:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T12:47:57.474Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 15, 2020 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:01.276Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 12:48:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:14.101Z: Workers have started successfully.
Apr 15, 2020 12:48:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:14.133Z: Workers have started successfully.
Apr 15, 2020 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:53.936Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:54.411Z: Cleaning up.
Apr 15, 2020 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:48:54.522Z: Stopping worker pool...
Apr 15, 2020 12:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:50:46.908Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 15, 2020 12:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T12:50:46.953Z: Worker pool stopped.
Apr 15, 2020 12:50:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-15_05_47_19-5946839451280560205 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2252893c-aab3-4f5c-a967-273e53250cc7 and timestamp: 2020-04-15T12:50:53.312000000Z:
Metric: Value:
read_time 23.391
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 55.624 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 45s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/cj5hhmb36yi6i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #386
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/386/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-9642] Fix infinite recursion.
------------------------------------------
[...truncated 282.90 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 15, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lchaQeFcvjLOrpjI6OJzIA.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-URf_xMw1CHU8V6_sCTlhMQ.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-k6Aki9M3khIu3qhkCPvJmA.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-G_Lg6PiJgGgR_xc2g69LtA.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-TRbs6U-3UHyZ695JVDai2w.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ql4PHgqpmXnWLAkr7AW0Mw.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-Dnryb-TGF03mFs6Cwp-Urw.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-9esVlERI_gDNZcQvT4TcRQ.jar
Apr 15, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bXYpv_4sxyumKcgADXBSlw.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-xDX4qAeTUJYNdNCm6Rl8RA.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-pLYiacN73wxEcffe-N4CEQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-GJPRWRiYMqE9lPqWgXVGRA.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-AD52WBabROYCzEs-m03pNQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-ySCKYL-WBdbUOYzYhkQSJQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-_TW4qBiC_FuvhTBNJZ3Q1Q.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-VLtR3XYygtfMeKntQ2UxtQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-TkflmdjPdEiEO-vpGZ616Q.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-U8t9f7fPj_kCxaas-yVMYA.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-31PPEyLgVuEHNYiAJ56d4w.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-R4o5dfydk8fd2znKyggLPA.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-Y-fP4bwBJlyM9jNZQ1ysGQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-foQiUdrJjZTmeJsugb611w.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-NPTYvdkD9226pMTcXhtGVg.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-kejLyzMuYEAW27B80W5ujQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-y455yhnfcKFKa6vD4TaBiw.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-3YGUgV-ePeJLOeCOTZL5GQ.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-g03Tpuv2moAHFCME7Q4S_Q.jar
Apr 15, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-g03Tpuv2moAHFCME7Q4S_Q.jar
Apr 15, 2020 6:45:31 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-g03Tpuv2moAHFCME7Q4S_Q.jar&uploadType=resumable&upload_id=AEnB2Upr_Pxil-Ojklqc2kGRggGSAIDUnsp2_plBD_jphgZ2zaeid8mBHNnktgrNN5vRW4loPGKfYvsE8kDIDYA03EB5at96Og.
Apr 15, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 15, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-g03Tpuv2moAHFCME7Q4S_Q.jar
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 15, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 15, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash 1Ztx9zCAlM702PyHhzDCaQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1Ztx9zCAlM702PyHhzDCaQ.pb
Apr 15, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 15, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-14_23_45_36-16366947139302386789?project=apache-beam-testing
Apr 15, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-14_23_45_36-16366947139302386789
Apr 15, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-14_23_45_36-16366947139302386789
Apr 15, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T06:45:36.598Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:39.907Z: Checking permissions granted to controller Service Account.
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:43.275Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:43.874Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:43.907Z: Expanding GroupByKey operations into optimizable parts.
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:43.934Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.002Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.027Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.049Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.076Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.330Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:45:44.398Z: Starting 5 workers in us-central1-c...
Apr 15, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:46:10.504Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:46:10.541Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 15, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:46:15.873Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T06:46:16.636Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 15, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:46:34.276Z: Workers have started successfully.
Apr 15, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:46:34.309Z: Workers have started successfully.
Apr 15, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:47:10.747Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:47:10.980Z: Cleaning up.
Apr 15, 2020 6:47:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:47:11.062Z: Stopping worker pool...
Apr 15, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:48:20.388Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 15, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T06:48:20.430Z: Worker pool stopped.
Apr 15, 2020 6:48:26 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-14_23_45_36-16366947139302386789 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 6d049d12-0c42-42c1-91be-ef236d91408a and timestamp: 2020-04-15T06:48:26.821000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.664
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 6.728 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 8s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/yplx7nkjma2iq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #385
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/385/display/redirect?page=changes>
Changes:
[github] [BEAM-9642] Add SDF execution units. (#11327)
[github] [BEAM-9136] reduce third_party_dependencies size (#11416)
[github] [BEAM-9746] check for 0 length copies from state (#11413)
------------------------------------------
[...truncated 285.00 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 15, 2020 1:04:54 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-5yLwebkAKBhM6aK8gbw5aw.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-fSBJRZidj7wlRXqCDYa6ag.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LVYoh8JS4eIqcJMX0hYxpQ.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-gXuHeJqOmdfbGU_44GU34w.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-k_8qK-QYdFC82QJlIHqkBw.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-8264eiTlqjizDN3J6mFp_g.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-dDVWUKo6eWPPiSq6w-IQKg.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-f6Drmn12z82qadfqHXQY_g.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-6822TCCVxP3xxV-8ZOcadA.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-ky6noYots0kz934ZtMrbjw.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xRuLtmAbvWlw_2RzRG5Iyg.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT--_lcDEsASO-sYuLzvjyQYw.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-QiaXP6fnd9kyhsm_BALwXQ.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-6oIjcTV2bUrBkPJjH4KIkQ.jar
Apr 15, 2020 1:04:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-ZDGojxU0sNTl-KU7c0LLdw.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-W_tir8WUjXLghqdZqiAhng.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-p7kFIUgObpKOoLohys9Ixw.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-B_y4cNpfCvM79kS8uSWHEg.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-QQ1yQZQyQ1-TkDlvpMD08Q.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-TzOichZXm69VYe-_xZUxcA.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-jg88Jj3ldpxu9UTg4uL_ig.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-WJVOpS87YlgTP-NFaGby1w.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-4lxY6MEmobBkASF7KHcTkA.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-yj8AhdvRltawp1H-7qT5fA.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-MRv3ql3va8Gq2jjgPn9Nqw.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-8se1DV_kPRkkqP4RvvuK0g.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-E9edtYCLoHFQXlP2uw9Oow.jar
Apr 15, 2020 1:04:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-E9edtYCLoHFQXlP2uw9Oow.jar
Apr 15, 2020 1:04:57 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-E9edtYCLoHFQXlP2uw9Oow.jar&uploadType=resumable&upload_id=AEnB2UoRJGafe6ZiYNSZIjbrt9q6oHfMi4Jj9DOzKja7B76cAITi_YFhzJgvpF5oLK25bWr_JKnbbzOjM64NUnQPQiQnbKlB66Ng4CQDhioUL7DZF6g5NGI.
Apr 15, 2020 1:04:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 15, 2020 1:05:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-E9edtYCLoHFQXlP2uw9Oow.jar
Apr 15, 2020 1:05:06 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 11 seconds
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash CneoWrpiNDxrunND0uHq3w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CneoWrpiNDxrunND0uHq3w.pb
Apr 15, 2020 1:05:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 15, 2020 1:05:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-14_18_05_07-4211506028660891972?project=apache-beam-testing
Apr 15, 2020 1:05:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-14_18_05_07-4211506028660891972
Apr 15, 2020 1:05:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-14_18_05_07-4211506028660891972
Apr 15, 2020 1:05:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T01:05:08.016Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 15, 2020 1:05:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:11.095Z: Checking permissions granted to controller Service Account.
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:15.701Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.369Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.406Z: Expanding GroupByKey operations into optimizable parts.
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.432Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.515Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.544Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.582Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.618Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:16.997Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 1:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:17.077Z: Starting 5 workers in us-central1-c...
Apr 15, 2020 1:05:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:43.329Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 1:05:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:43.370Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 15, 2020 1:05:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:05:48.809Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 15, 2020 1:05:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-15T01:05:50.669Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 15, 2020 1:06:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:06:02.054Z: Workers have started successfully.
Apr 15, 2020 1:06:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:06:02.096Z: Workers have started successfully.
Apr 15, 2020 1:06:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:06:35.247Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 15, 2020 1:06:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:06:35.455Z: Cleaning up.
Apr 15, 2020 1:06:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:06:35.531Z: Stopping worker pool...
Apr 15, 2020 1:08:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:08:04.790Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 15, 2020 1:08:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-15T01:08:04.829Z: Worker pool stopped.
Apr 15, 2020 1:08:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-14_18_05_07-4211506028660891972 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c233b3a7-105a-4fdc-9dee-d6aec195c59b and timestamp: 2020-04-15T01:08:12.641000000Z:
Metric: Value:
read_time 12.858
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 27.002 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 45s
101 actionable tasks: 70 executed, 31 from cache
Publishing build scan...
https://gradle.com/s/ecwyrfgzhgowi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #384
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/384/display/redirect?page=changes>
Changes:
[xhan] Documentation bug fix for FlatMapElements#via() SimpleFunction in the
[github] [BEAM-7923] Screendiff Integration Tests (#11338)
------------------------------------------
[...truncated 284.38 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 14, 2020 7:15:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 7:15:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 14, 2020 7:15:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 7:15:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 14, 2020 7:15:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 14, 2020 7:15:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 14, 2020 7:15:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 14, 2020 7:15:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-4p6qETwHXtOIWzXmyaAqHw.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-cTcJXmb9NKvCKNhbdEGxsQ.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-eUvRwKdbx3qVGiJSrosMIQ.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-wFwKqFEQ5FLVsOk4i9p5Xg.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-6vkyPEl7fyJnU47tbG1kqw.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-s1l6tNRlSIam1ygf8lDKQA.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-U_LwnvLpfO1L2dfYtZHyLw.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Pd5bJ7DctsBceAfv-7-6ng.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-dEsMbe17aK_TYLBmdBcx3Q.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-07jTbWQ9DL5ITfAXdr-YuA.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-NRrZWXPwY5xIX0IHu5sDlQ.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-EQKuBGAHU3Ux8J-tleRWXA.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-fR4cWEQYJ-cuX06YkRyg8g.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-qBEcD2gRPzaWTJLjFin91Q.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-fDIFGm6mkJ2gs7tDcC5j8w.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-l9GBw-28xzYz3cyZPeqIKg.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-TPMYwgbbCeuemsIuEX2TTg.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-TSqIzT4OskYnR_PM6kf6aA.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-HyvQGXuMCAMsZLXAVsB2Nw.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-y219lhyLX4hm6W8geSKN8w.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-sKLrv_GxWZPnfLd1XLOWJw.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-VlU_6wdbd3rymBfi94KJZg.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ctX599SEvFveZf-XaYBzXg.jar
Apr 14, 2020 7:15:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-vD5J641lX3C9-8tclb10hA.jar
Apr 14, 2020 7:15:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HL-KO43V10dUDFbT3OyV0w.jar
Apr 14, 2020 7:15:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-dVwbTsGHtGWbqdXPZ-rsnQ.jar
Apr 14, 2020 7:15:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-eXHs6DXxD6hCGTTiQVNOig.jar
Apr 14, 2020 7:15:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HL-KO43V10dUDFbT3OyV0w.jar
Apr 14, 2020 7:15:38 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HL-KO43V10dUDFbT3OyV0w.jar&uploadType=resumable&upload_id=AEnB2UoMH297ebPYrDuB14SykoZv5p7sbYPHB0_1pe0i1WzSlhDyKU15agR0Xrl5P23QwXGcKAkf3tyVaOCU9dxOcgXPlpBt3uTyy-pRFkE_G2ba9ow8v1w.
Apr 14, 2020 7:15:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 14, 2020 7:15:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-HL-KO43V10dUDFbT3OyV0w.jar
Apr 14, 2020 7:15:44 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 8 seconds
Apr 14, 2020 7:15:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 14, 2020 7:15:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 14, 2020 7:15:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 14, 2020 7:15:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 14, 2020 7:15:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 14, 2020 7:15:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 3vLj0L2bj7wysSBZsFNfmQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3vLj0L2bj7wysSBZsFNfmQ.pb
Apr 14, 2020 7:15:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 14, 2020 7:15:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-14_12_15_46-12697262194285339994?project=apache-beam-testing
Apr 14, 2020 7:15:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-14_12_15_46-12697262194285339994
Apr 14, 2020 7:15:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-14_12_15_46-12697262194285339994
Apr 14, 2020 7:15:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T19:15:46.106Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 14, 2020 7:15:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:49.253Z: Checking permissions granted to controller Service Account.
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:54.212Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.082Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.122Z: Expanding GroupByKey operations into optimizable parts.
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.152Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.238Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.272Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.297Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.331Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 14, 2020 7:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.808Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 7:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:15:55.871Z: Starting 5 workers in us-central1-c...
Apr 14, 2020 7:16:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:16:23.278Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 7:16:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T19:16:26.168Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 14, 2020 7:16:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:16:42.356Z: Workers have started successfully.
Apr 14, 2020 7:16:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:16:42.390Z: Workers have started successfully.
Apr 14, 2020 7:17:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:17:21.754Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 7:17:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:17:21.946Z: Cleaning up.
Apr 14, 2020 7:17:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:17:22.035Z: Stopping worker pool...
Apr 14, 2020 7:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:18:47.303Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 14, 2020 7:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T19:18:47.343Z: Worker pool stopped.
Apr 14, 2020 7:18:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-14_12_15_46-12697262194285339994 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ff966e26-3e2e-4f6d-b544-d951e63a8d79 and timestamp: 2020-04-14T19:18:53.762000000Z:
Metric: Value:
read_time 15.72
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 28.119 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 32s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/kz7spjkrww3ie
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #383
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/383/display/redirect?page=changes>
Changes:
[github] [BEAM-9650] Add PeriodicImpulse Transform and slowly changing side input
------------------------------------------
[...truncated 282.77 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 14, 2020 12:51:07 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Wt9xaBYanMtlIwHz3hn2Rg.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-4hKAq-5PDQ9uTHmAC1pLwg.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-O-DTZwFzAgOisog1cjSltA.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-ePHOojqJYzYUnE5F8c30Iw.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-dfprIBntYy6Mku4gdZtB3g.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-vRVVRgjqokFOjY3lB3tv1g.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-EO-f239PN4QJybt0iPCLJQ.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-NfQ8uKU7A2DCb67GMLx9Pg.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-E0dIOO_frSM5Hso2Vsj8jQ.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-Ed2nPXxqyQAf81C7evwblQ.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-uYSe0brWbyKI9ImhEFqyoA.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-J25NrUbMW4Rd10pakfVA9w.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-w36ESXPerOZ89t1W3kzcqw.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Kj1xzhubLMR99R1EwHOlJQ.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-aB9jJlzmX5u-gSJi2pvsag.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-uKnR2zBigQCJ-F0WWPts9A.jar
Apr 14, 2020 12:51:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-eKFdUuML_Si79bhaSTecsA.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-4dmXw1t7K8ui_v5TavETKA.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-jh2-mEtGGDJWXjcz8pphmQ.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-U3ujY5p_-XzucydrOJvyVQ.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-VMS41eT771JQZc6RWHIVyw.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-opg2E4AhBkydvQKHJ8lF2g.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-y1gdsbb9iqy87eV9QXPs5Q.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-pKU3VfD58jEd1ibDVBEwPA.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-PuUoN8P61SsGMy64vcL1gQ.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-pt7f4SnmqWAbD2tF24iLrw.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-pKU3VfD58jEd1ibDVBEwPA.jar
Apr 14, 2020 12:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-N1-wOPI4Ks9gSpUVSIoDLg.jar
Apr 14, 2020 12:51:10 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-pKU3VfD58jEd1ibDVBEwPA.jar&uploadType=resumable&upload_id=AEnB2UrjPwJbJK4fFZrNcstdsuVRctBW6nDVeJI8GUjbzPVSewz8vksf6qCxoeL05Q27qxeG2pK7vIH_-NyzhZn4kNODo8zPag.
Apr 14, 2020 12:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 14, 2020 12:51:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-pKU3VfD58jEd1ibDVBEwPA.jar
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 7 seconds
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 14, 2020 12:51:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 14, 2020 12:51:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12100 bytes, hash Az4236-om10xs4S6BQjl8w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Az4236-om10xs4S6BQjl8w.pb
Apr 14, 2020 12:51:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 14, 2020 12:51:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-14_05_51_16-9175935564495312043?project=apache-beam-testing
Apr 14, 2020 12:51:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-14_05_51_16-9175935564495312043
Apr 14, 2020 12:51:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-14_05_51_16-9175935564495312043
Apr 14, 2020 12:51:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T12:51:16.795Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 14, 2020 12:51:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:20.709Z: Checking permissions granted to controller Service Account.
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.101Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.669Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.706Z: Expanding GroupByKey operations into optimizable parts.
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.727Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.807Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.828Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.852Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:25.887Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:26.312Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 12:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:26.377Z: Starting 5 workers in us-central1-f...
Apr 14, 2020 12:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T12:51:47.588Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 14, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:51.717Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 12:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:51.748Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 14, 2020 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:51:57.151Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 12:52:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:52:09.820Z: Workers have started successfully.
Apr 14, 2020 12:52:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:52:09.859Z: Workers have started successfully.
Apr 14, 2020 12:52:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:52:45.702Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 12:52:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:52:45.888Z: Cleaning up.
Apr 14, 2020 12:52:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:52:45.955Z: Stopping worker pool...
Apr 14, 2020 12:54:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:54:47.886Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 14, 2020 12:54:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T12:54:47.934Z: Worker pool stopped.
Apr 14, 2020 12:54:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-14_05_51_16-9175935564495312043 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9ae8b5ff-327d-421f-b7c3-f30bf6d77e49 and timestamp: 2020-04-14T12:54:55.692000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.138
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.038 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 56.66 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 56s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/euyo7nmat732a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #382
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/382/display/redirect?page=changes>
Changes:
[github] Merge pull request #11389 from Refactor the BCJ and capture controls to
[github] [BEAM-i9751] upgrade zetasql to 2020.04.1 (#11410)
------------------------------------------
[...truncated 283.16 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 14, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 14, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 6:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 14, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 14, 2020 6:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-YLSrSMmqBWLs9z--bufYNA.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test--cL-sl9YvQbeTTiPxLnr4g.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-69gysz7kXDVA_8SDVxi3lQ.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Jqx_yjkBZI_Fowa1appu0A.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-nbiIcpMNG8rWTAlSAAizXA.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Xdndv4H1mcAslx3lAxKR3w.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-J2_qHJpN4fzkjvOrbouemg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-4TivNXACZyXu6-zBJU56vA.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT--3viJXrGjdRSL4DEttvYvg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-VOEyOLm-bnpdBM6qfrF7ew.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-Iaxip77op8yj7pBBjWWIfg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-kyKuPH0_T-pXFfvAEMn13g.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-glwAiFOl6z2gA65xlmcYvg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-uxLER2MNhlWKaYtGE3qnAQ.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-O93u6HQ2-SjvcMShn1UUrg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-B07G5BUbJDpi62B6GTWgcg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-RYi2HX6_gr39HENLzxFhQw.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-a0mtrCD_XeOa2scmGuehlg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-8KgP6T3RHoEDavpjBrfzcg.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-_2IzfWVd4-puDwdr3zYIGA.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-8uevUwc-4Opdx9JVRBzu9A.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-hVrZ--Htcf_rWLHSP2nfbA.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-_ir7lsEGWvAfOlUknBPEfw.jar
Apr 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-H3QpfQgW8zu0GHf6ebk-NA.jar
Apr 14, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-H3QpfQgW8zu0GHf6ebk-NA.jar
Apr 14, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-gmG6qjMG9fsFT7HONYFFgA.jar
Apr 14, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-IkLJCfyTd-n_fs7aFOVjGQ.jar
Apr 14, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-MatGQLCVLw5Kl68evrDlBQ.jar
Apr 14, 2020 6:45:32 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-H3QpfQgW8zu0GHf6ebk-NA.jar&uploadType=resumable&upload_id=AEnB2Uo9SB6tEG0mdzXqgbNgU_PF37oFctyPQQz5XtvNTIXcyuIswnegSJQEbJualJs42R6yfEa2dIyh_b3FDrC9ZKzI2bwOrg.
Apr 14, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 14, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-H3QpfQgW8zu0GHf6ebk-NA.jar
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 10 seconds
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 14, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 4hMXT1cgQVk6zWS5dhOv-A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4hMXT1cgQVk6zWS5dhOv-A.pb
Apr 14, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 14, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-13_23_45_41-5657267475558130253?project=apache-beam-testing
Apr 14, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-13_23_45_41-5657267475558130253
Apr 14, 2020 6:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-13_23_45_41-5657267475558130253
Apr 14, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T06:45:41.383Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 14, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:45.312Z: Checking permissions granted to controller Service Account.
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:48.445Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.035Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.077Z: Expanding GroupByKey operations into optimizable parts.
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.107Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.195Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.232Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.262Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.300Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 14, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.653Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:45:49.740Z: Starting 5 workers in us-central1-f...
Apr 14, 2020 6:46:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T06:46:03.778Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 14, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:46:12.970Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:46:13.007Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 14, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:46:18.409Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:46:34.912Z: Workers have started successfully.
Apr 14, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:46:34.952Z: Workers have started successfully.
Apr 14, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:47:08.523Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:47:08.731Z: Cleaning up.
Apr 14, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:47:08.814Z: Stopping worker pool...
Apr 14, 2020 6:49:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:49:06.297Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 14, 2020 6:49:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T06:49:06.341Z: Worker pool stopped.
Apr 14, 2020 6:49:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-13_23_45_41-5657267475558130253 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9f941ede-bd0e-4b23-a1a8-7de5c7c678e1 and timestamp: 2020-04-14T06:49:12.992000000Z:
Metric: Value:
read_time 15.502
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 52.733 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/uvfcm3xv277ku
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #381
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/381/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9496] Evaluation of deferred dataframes via Beam operations.
[robertwb] Fix and test tuple inputs and outputs.
[robertwb] Comments and clarification.
[kcweaver] [BEAM-9744] Add missing region option to py perf tests.
[lcwik] [BEAM-9562] Fix output timestamp to be inferred from scheduled time when
[kcweaver] [BEAM-9744] Remove --region option from SQL tests.
------------------------------------------
[...truncated 282.47 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 14, 2020 12:46:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 12:46:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 14, 2020 12:46:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 14, 2020 12:46:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 14, 2020 12:46:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 14, 2020 12:46:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 14, 2020 12:46:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 14, 2020 12:46:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-O-sqcTtixKLTj21VsZS3aA.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-DjobiDCF6jv-HlpSs0rSdQ.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-aQ_4aexMgr9etFfFrbkT-Q.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-Y257qrjqTJAanR4cGkZ8IQ.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-83L6yNFUCjgzRR-fSeECVg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-1jZaITyNU4EO0eBy_EPe7A.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-6zqu88pbdr1U3DRT3OZPxw.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-MtVGgL73NqlRxLC_ktW5cg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-Xd8sf5wmOFHSKGg0Ja0I_w.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ukxhVEkrf1zmi9Rc198z4w.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-I0u8vdw77Si5vyXEhKgmyQ.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-ITTL1ZPr6Vi8Tu1tGrhrUg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-fPo5Xh3MNAUlyZJ2vRGP3w.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-XNk-rdy6eNTI41nAM98WTw.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-VQrB1bPvvNKxclg8i3ecOg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-eSoxSMvrce2lMmAXUoBaFg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-fFENPsz0tPKiF0g-UDS-wg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT--4O_M0SunE4i_wEWHygGWg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-qI1YlLSAmEsMyn5i-0yGDw.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-KxNIwz3uPWcWQn9VYuaCIw.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-7BE_QuPhtO46jceGuE_qvA.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-RN9o9hnxJpmNyQtUPMXGTg.jar
Apr 14, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-phn_n2SXBMy-Dl5Fk-tKJw.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gUfD_8B-9iJ2ueJBkff7aQ.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8q0D-EmfnX4JFCtrijP0ZA.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-JTmvO7pUOj7yXU3oxxwOVQ.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8q0D-EmfnX4JFCtrijP0ZA.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-kBfol6lw7strc5RG2j9nSw.jar
Apr 14, 2020 12:46:39 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8q0D-EmfnX4JFCtrijP0ZA.jar&uploadType=resumable&upload_id=AEnB2UrBLK8sp2-5GWwacIY_hkwrIo3Oq5T1XIM-Dd34HY5mf0tolmiOTJT9Ozt-tZZ1K90IW95zztOojbIMa7a4d2gHpNXv7y94ILdpoiqY-PozCutVPQY.
Apr 14, 2020 12:46:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 14, 2020 12:46:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-8q0D-EmfnX4JFCtrijP0ZA.jar
Apr 14, 2020 12:46:43 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash X7nrzvkZrRJ3eu2pp6u16A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-X7nrzvkZrRJ3eu2pp6u16A.pb
Apr 14, 2020 12:46:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 14, 2020 12:46:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-13_17_46_45-11050409604923450417?project=apache-beam-testing
Apr 14, 2020 12:46:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-13_17_46_45-11050409604923450417
Apr 14, 2020 12:46:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-13_17_46_45-11050409604923450417
Apr 14, 2020 12:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T00:46:45.093Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 14, 2020 12:46:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:48.779Z: Checking permissions granted to controller Service Account.
Apr 14, 2020 12:46:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:52.316Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:52.858Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:52.893Z: Expanding GroupByKey operations into optimizable parts.
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:52.922Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.031Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.069Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.128Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.165Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.607Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 12:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:46:53.695Z: Starting 5 workers in us-central1-f...
Apr 14, 2020 12:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-14T00:47:02.618Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 14, 2020 12:47:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:47:19.133Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 14, 2020 12:47:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:47:35.168Z: Workers have started successfully.
Apr 14, 2020 12:47:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:47:35.213Z: Workers have started successfully.
Apr 14, 2020 12:48:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:48:08.570Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 14, 2020 12:48:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:48:08.863Z: Cleaning up.
Apr 14, 2020 12:48:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:48:08.958Z: Stopping worker pool...
Apr 14, 2020 12:50:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:50:21.469Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 14, 2020 12:50:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-14T00:50:21.529Z: Worker pool stopped.
Apr 14, 2020 12:50:29 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-13_17_46_45-11050409604923450417 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a45c934e-7ab1-4645-96ae-08883ce3aeb1 and timestamp: 2020-04-14T00:50:29.295000000Z:
Metric: Value:
read_time 14.416
fields_read 4375276.0
Gradle Test Executor 402 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.001 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.002 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 1.738 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/qoxvqc27yxkmu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #380
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/380/display/redirect?page=changes>
Changes:
[suztomo] google-api-client 1.30.9
[chamikara] Updates Dataflow stateful DoFn setup to support external transforms
[github] [BEAM-9738] Update dataflow to setup correct docker environment options.
[github] [BEAM-9136]Add licenses for dependencies for Java (#11243)
------------------------------------------
[...truncated 284.82 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 13, 2020 6:51:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 13, 2020 6:51:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 13, 2020 6:51:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-VX-lPzbVdhjxe2SeT_-LVg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-WGqSe3vMyzaJHY2Rf_i0NA.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Qr_OJr4WM5tgp5Tz-x5gnQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-jNjloWbx1mLLdifPDe_tQg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-4zwGmTtZVrmxKkAfVzPJrw.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-c6xwz8c2UxHHqmRQvu5zyQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-AduF0MgmqhIRjgjnlJh3fQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-v6-f_rxNC2z7lz8fknzQlA.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-kFcBMB45MZ-VetZYdeIhMA.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-r4IinDrjxb3oCR3fX7Rbew.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-3zbT_d1oUbQUYrYbHur3Tw.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-cTOTHFilSB1Y1g2ZkFwIdQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-E30_gZLr7kg8cU8aS8TlBg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-H5PABOodimVpDVPkYSllFQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-J8HRG6aR-b4EE5roJ0n4ZA.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-PSWmOe8nhbWdA94TBChYMg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test--uThClZNtJ9lfNiSHcbCZw.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-rpVh_IHxzTyT4pVz1ZUP4A.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-UFb2wEc3JAMJsP0ZePQ-rg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Wngf4Z4ygeyQPtUeK08Osg.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-_gJNVwS8gc3SeJ4hDfgywQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-kOb2zoEQhVSfDXq1TjpT4Q.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-VQV-kGIhjzUlNFhIzVtugA.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-vUpFOOZk7FdFB0qyifdXVQ.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-YXwcBRAf-EcEhXOBFdzKnw.jar
Apr 13, 2020 6:51:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AbQRTyBoLjuC2i1St7LrNg.jar
Apr 13, 2020 6:51:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AbQRTyBoLjuC2i1St7LrNg.jar
Apr 13, 2020 6:51:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-MmBevNUCxg6hk3AQHzDTsg.jar
Apr 13, 2020 6:51:21 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AbQRTyBoLjuC2i1St7LrNg.jar&uploadType=resumable&upload_id=AEnB2UpMRwr0XK8TpUegakauXt_CH7AbtpmuX4doifZVEVt_pyrKo2dS7d_SxG4KHp0wQmz-UhPtdp9fOA1w9xiiwro7EOrKqQ.
Apr 13, 2020 6:51:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 13, 2020 6:51:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-AbQRTyBoLjuC2i1St7LrNg.jar
Apr 13, 2020 6:51:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 8 seconds
Apr 13, 2020 6:51:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash FPgwNI-3DNxnxhDJqTVZWQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-FPgwNI-3DNxnxhDJqTVZWQ.pb
Apr 13, 2020 6:51:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 13, 2020 6:51:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-13_11_51_29-2275682355150010120?project=apache-beam-testing
Apr 13, 2020 6:51:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-13_11_51_29-2275682355150010120
Apr 13, 2020 6:51:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-13_11_51_29-2275682355150010120
Apr 13, 2020 6:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T18:51:29.762Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 13, 2020 6:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:32.837Z: Checking permissions granted to controller Service Account.
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:37.367Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.009Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.055Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.087Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.182Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.223Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.249Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 13, 2020 6:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.279Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 13, 2020 6:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.840Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 6:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:51:38.930Z: Starting 5 workers in us-central1-c...
Apr 13, 2020 6:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T18:51:43.765Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2020 6:52:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:52:05.659Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 6:52:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:52:05.691Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 13, 2020 6:52:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:52:11.084Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 6:52:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:52:25.223Z: Workers have started successfully.
Apr 13, 2020 6:52:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:52:25.260Z: Workers have started successfully.
Apr 13, 2020 6:53:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:53:02.638Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 6:53:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:53:02.835Z: Cleaning up.
Apr 13, 2020 6:53:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:53:02.907Z: Stopping worker pool...
Apr 13, 2020 6:54:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:54:22.676Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 13, 2020 6:54:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T18:54:22.716Z: Worker pool stopped.
Apr 13, 2020 6:54:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-13_11_51_29-2275682355150010120 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0bd67300-b0f3-4a48-8a22-a7a5c419d40b and timestamp: 2020-04-13T18:54:30.377000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.92
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 21.172 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 44s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/z6chanj7xxdq4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #379
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/379/display/redirect>
Changes:
------------------------------------------
[...truncated 283.43 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 13, 2020 12:55:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-FSlOb5bYmCXMkm6B6nGTEg.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lVNmu5_H5aDn06btbqfQhQ.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-5sUr2Rf8iRSOZn5arIUsQQ.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-2yv5Etr14gvpx_5PmgUyOg.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-V2K8-yLQfAfaOqsPvPtN8g.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-LK35WGmsctEgtpi-mBZeFA.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-3WYIlMocLE7vaSB6T-6dcA.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-t-ltxL4kIPXw8Np30JR2mg.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-PPWp0xW4pWJbZh03WFHYZg.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-RtHgmsUpcg4J6J5hxj_H7g.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-uNBse41GXuvmxP1yiSp3Gw.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-gRVM4EEHwrRNrB2Gv8DgQA.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-34uz9SYDxWy-bG6oQY5zlA.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-8FtxU1GSv8XHTITQ0RTBLQ.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-t_u6eubjZZNn5DvPgYZJ5A.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-MZxnRaemxB4EBGz4PUiTxA.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-SYUzD227NZMZ1oDLQQYAlQ.jar
Apr 13, 2020 12:55:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Rk3Wkmk2k49g-PPWxx1-DA.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-Ji3YjVTUCxc_S8hFE7JXfA.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-wUJ-fKAMQDsQyi-fxXlvVw.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-OAdJ7z-v3tKc671K_8yCsw.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-GLPgHI_n-25WberTz20DXQ.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-dcmJd4JxLh31i0zQg05brg.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-z4StrOELvRU3iY8jTy08uQ.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-dnBU_aiCFMEEpch7mqCegQ.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Jw2MFRm2sdzo4TpRSe8L6g.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-2trsowB1y8mGefMaeha21g.jar
Apr 13, 2020 12:55:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Jw2MFRm2sdzo4TpRSe8L6g.jar
Apr 13, 2020 12:55:28 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Jw2MFRm2sdzo4TpRSe8L6g.jar&uploadType=resumable&upload_id=AEnB2Uqd3V2gaLoaJtBFxGD2DwQ7H_M_qoTc81EqnARLaYUR24dkkDzGiZDBhaC4Zrl8k2tzK-irc4BiD4Qu_x7hHamjjDOAXA.
Apr 13, 2020 12:55:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 13, 2020 12:55:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-Jw2MFRm2sdzo4TpRSe8L6g.jar
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 8 seconds
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 13, 2020 12:55:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 5n2LuNKDteW0sqLoac5YqQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5n2LuNKDteW0sqLoac5YqQ.pb
Apr 13, 2020 12:55:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 13, 2020 12:55:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-13_05_55_36-14971669735868127066?project=apache-beam-testing
Apr 13, 2020 12:55:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-13_05_55_36-14971669735868127066
Apr 13, 2020 12:55:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-13_05_55_36-14971669735868127066
Apr 13, 2020 12:55:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T12:55:36.194Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 13, 2020 12:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:39.447Z: Checking permissions granted to controller Service Account.
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:42.807Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.382Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.426Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.466Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.559Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.592Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.621Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 13, 2020 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:43.658Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 13, 2020 12:55:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:44.077Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 12:55:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:55:44.164Z: Starting 5 workers in us-central1-c...
Apr 13, 2020 12:56:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:56:11.297Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 12:56:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:56:11.321Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 13, 2020 12:56:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T12:56:13.755Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2020 12:56:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:56:16.717Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 12:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:56:32.451Z: Workers have started successfully.
Apr 13, 2020 12:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:56:32.493Z: Workers have started successfully.
Apr 13, 2020 12:57:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:57:08.118Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 12:57:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:57:08.339Z: Cleaning up.
Apr 13, 2020 12:57:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:57:08.433Z: Stopping worker pool...
Apr 13, 2020 12:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:58:44.787Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 13, 2020 12:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T12:58:44.837Z: Worker pool stopped.
Apr 13, 2020 12:58:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-13_05_55_36-14971669735868127066 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a2e7e8d6-fea6-4a01-ac3f-65265cd47bd2 and timestamp: 2020-04-13T12:58:52.017000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.628
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 35.301 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 36s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ac2awwojby5ce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #378
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/378/display/redirect>
Changes:
------------------------------------------
[...truncated 283.99 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 13, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 13, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 13, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 13, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 13, 2020 6:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 13, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-ohT_NA2utigZBbdQ_-1DXA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-W-b0Yq9UhpQvWRJq0rethg.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-zXN9ALFTGYt3WJx77xqLvA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-2--A643Erp0U7ylkkligqw.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-SCdX83MZU0IQxNPGp8beVA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-g8JQTdIOtLnMy0ZRV0jN0A.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-V1ohTgnqa_V7z1w6TXjnSQ.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-PzVeE-zKPulA2xzfQxKTsA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-podCK_7aAQYijOerCjzOtA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Ksm1cpTLTxBzzGqge3KgFg.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-yW1nfUaZVV9QxRMVr8O7WQ.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-XbcFYcHFD-UoFA9B74_0eA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-wCr_Vpl9oJpMR3FYkVJrpg.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-5Ad0NeIp07k11-zr1XU9Ow.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-1Rdt5BN4WRSaubW_LnYZzA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-GQB-xXkXnr5N2jnnXwVgKA.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-IFtJ2bR4Q9EmPZu_VDtdEw.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-eMxMD5hjNgHfdxOJzxrf0Q.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-02FzDKCX2YLWpIRDRjUOjw.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-1MJVlQ2lWAsHB9MXChR2ug.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-pX4PemcvfLRqrMWgQnSTyQ.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-_t6g-Qzkkk7m9bQquOMslw.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-uHCOupTl3BJaYy2Bec4b1w.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-FpxxLbK9bi8qnVCy_klIOQ.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-gtLjod-lwBBZ99DUdqfNZQ.jar
Apr 13, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-ahnBcwtxwejohI7qbloS1A.jar
Apr 13, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-241kHA5IpihWMyP4TSDqlg.jar
Apr 13, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-241kHA5IpihWMyP4TSDqlg.jar
Apr 13, 2020 6:45:20 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-241kHA5IpihWMyP4TSDqlg.jar&uploadType=resumable&upload_id=AEnB2Uo0Hy2jEjP469WoTcYI1AOMLAmoZApdSaNkYsDul58PVfbl2S286q6oj3x0x6OwLmUHBPR5wo_nWTkDp4VMna5zOUWLcqGVMwG0NtmuHtoW-DczHsY.
Apr 13, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 13, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-241kHA5IpihWMyP4TSDqlg.jar
Apr 13, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 9 seconds
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash r2S1DrReqd_l4g_Je5llIw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-r2S1DrReqd_l4g_Je5llIw.pb
Apr 13, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 13, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-12_23_45_28-1604925928851007924?project=apache-beam-testing
Apr 13, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-12_23_45_28-1604925928851007924
Apr 13, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-12_23_45_28-1604925928851007924
Apr 13, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T06:45:29.027Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 13, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:32.470Z: Checking permissions granted to controller Service Account.
Apr 13, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:36.629Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 13, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.185Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.262Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.300Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.389Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.418Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.450Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.483Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 13, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.881Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:45:37.962Z: Starting 5 workers in us-central1-f...
Apr 13, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T06:45:52.813Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2020 6:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:01.602Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 6:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:01.641Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 13, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:07.027Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:23.242Z: Workers have started successfully.
Apr 13, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:23.278Z: Workers have started successfully.
Apr 13, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:57.508Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:57.699Z: Cleaning up.
Apr 13, 2020 6:46:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:46:57.789Z: Stopping worker pool...
Apr 13, 2020 6:48:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:48:36.694Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 13, 2020 6:48:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T06:48:36.743Z: Worker pool stopped.
Apr 13, 2020 6:48:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-12_23_45_28-1604925928851007924 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 880aa508-e08c-4bc7-afd1-e1f7468978c2 and timestamp: 2020-04-13T06:48:43.526000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.284
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 35.855 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/wdvxe4e5uwqri
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #377
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/377/display/redirect>
Changes:
------------------------------------------
[...truncated 283.18 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 13, 2020 12:45:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-etCeWw0Y4PhbHcLttJqPtg.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-MT5WcKx4C8GL8dsh67l5Og.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-pAvfpV12s3Gitp2z0q1tIw.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-fpBZhGqQYYyJdnVXy9KOgg.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-JwrtoRgAyK1W-7i_VbIpcw.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-shTN464LwnvIfl7RIsIjGw.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-VbUiT7nPlZnB6d2L1B75gQ.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-GEwia3-E02qOIpV8vb8Cbw.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-prhvmVLcAFnsm_OyxjEBtg.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-vV38TClfTz6gduSk9jeH4g.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-dKI3MJ-xh2FmsGqu6jQSsg.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-OFiBXhzqmf_XNN8nUrYo3g.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-jJg0WfpOB1xe5aOLCB_PiA.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-huLErKDg0F0kBowS9zz0pw.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-BK5h58bLsg4pcbNigT7W2A.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-9oJ2XMyy-NYyToHh4Cgtbg.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-4PAvsww3CpCDJJumToefVQ.jar
Apr 13, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-85_fPSrsiogWnznLb9xghw.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-aNPVGyn08spL7bLPqfnKAw.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-uNUVHkyQjqlXvm3DxhEDsA.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-5ulAdcLzxu1vqpWhhVDSYQ.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-DxJxmOqrFovLfbfwyDZt_A.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-Cu_dEiBluNeix9HFSb6v-Q.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-SHvReoUzhgcwNpj7bJnVlQ.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-rBbfgVPuCf43xI6vLXUmug.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-9-uJrEBO5zPBy9c8qEwf8A.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VE80Dvzu4ktURFLbTqnURw.jar
Apr 13, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VE80Dvzu4ktURFLbTqnURw.jar
Apr 13, 2020 12:45:35 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VE80Dvzu4ktURFLbTqnURw.jar&uploadType=resumable&upload_id=AEnB2UoygTDl6buNCyBUmdAp_742PEWxjAlF-A2OTpUzoFruc8jrnEri69nWFXwaw1LWsNvMLImKUTSSqeAqtoGlgZuCb8GHIkhTludN1QVGAiFTKvULebc.
Apr 13, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 13, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VE80Dvzu4ktURFLbTqnURw.jar
Apr 13, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 7 seconds
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 13, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12098 bytes, hash ak_gLW_sD54-p_a4XVzX9w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ak_gLW_sD54-p_a4XVzX9w.pb
Apr 13, 2020 12:45:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 13, 2020 12:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-12_17_45_42-7253814715184459630?project=apache-beam-testing
Apr 13, 2020 12:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-12_17_45_42-7253814715184459630
Apr 13, 2020 12:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-12_17_45_42-7253814715184459630
Apr 13, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T00:45:42.104Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 13, 2020 12:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:45.111Z: Checking permissions granted to controller Service Account.
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:48.431Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:48.990Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.025Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.055Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.134Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.162Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.192Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.228Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.843Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:45:49.918Z: Starting 5 workers in us-central1-f...
Apr 13, 2020 12:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:46:12.938Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2020 12:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-13T00:46:19.316Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2020 12:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:46:35.086Z: Workers have started successfully.
Apr 13, 2020 12:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:46:35.116Z: Workers have started successfully.
Apr 13, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:47:07.356Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 13, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:47:07.568Z: Cleaning up.
Apr 13, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:47:07.654Z: Stopping worker pool...
Apr 13, 2020 12:48:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:48:51.434Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 13, 2020 12:48:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-13T00:48:51.479Z: Worker pool stopped.
Apr 13, 2020 12:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-12_17_45_42-7253814715184459630 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 017e4caf-55c0-4625-9afe-ee44477fdbdc and timestamp: 2020-04-13T00:48:57.465000000Z:
Metric: Value:
read_time 15.028
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.237 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 34.336 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/4qxnd4salxa6c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #376
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/376/display/redirect>
Changes:
------------------------------------------
[...truncated 282.26 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 12, 2020 6:45:31 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-BzqkVxsFohpUIdQ7BWmGSA.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-g3PgbJKbJXrjCZq8tXi3Ug.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-2b6_m-PAP5GUmBSMeYjWBg.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-D_ADQnTSpM9v3qwdlG17hg.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-AOKDi3LBIxcQgNXIhAotbw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-OQdCO7rjJFFnzHkdr9q8pg.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-fvMY24R7u8pOp9x6li5vcA.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-UcgAFKmgep9-uoSqcYtTPw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-PQbGIV0r3y2feTXw2VGzHQ.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-T5-8eJDLJWkPpMbBeYktcw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-OMCcUwOYsvDZPQ2CAKbcaA.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-2eUQCDV_R3v96BX1QLAkLw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-UQCxDTiTx_l3HAd5QoPfdw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-jddPeZ8SXKq9Hb2MNGCbCg.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-xFmQq9viR0ldz2bdosYuWw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-C9QSW_OF5_x_N8idUBd-hQ.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-bjZPuHKLDUhbv5eiwFXrOQ.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-ZAI43kd0MtB1LXvmLn_l4A.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-zY1zF-5z3zdAk4AFbutBVQ.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-clKSL7J6wXezge2nBnL-EA.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-zLbogATJzJ2zHaQKK99ycw.jar
Apr 12, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-x-eIg2umaElbaonB3Udx0A.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-c1V_H0PSwuS2rBXJAU8O1w.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-LtkgEuTbSFPT3-uUVrn1KQ.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cksC4Cm2gENiB7X1nPdB3w.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cksC4Cm2gENiB7X1nPdB3w.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-WO9mjnUG3BA3DaojjAfSdw.jar
Apr 12, 2020 6:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-86LTP_HYKWXiS7EvWLy7XA.jar
Apr 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cksC4Cm2gENiB7X1nPdB3w.jar&uploadType=resumable&upload_id=AEnB2UoxHSUb1JWmh0CgktiQ5sM0VX2bkW8wr70vv0tcA8z9yRDML2st-IzZCc-tmQwQcwPEIVJJqrRTv5hXzyGJ7qXy9exqZQ.
Apr 12, 2020 6:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 12, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-cksC4Cm2gENiB7X1nPdB3w.jar
Apr 12, 2020 6:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 8 seconds
Apr 12, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash q6bF6SyOpRY-s1k0c9x7YA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-q6bF6SyOpRY-s1k0c9x7YA.pb
Apr 12, 2020 6:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 12, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-12_11_45_41-10608369291339296041?project=apache-beam-testing
Apr 12, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-12_11_45_41-10608369291339296041
Apr 12, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-12_11_45_41-10608369291339296041
Apr 12, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T18:45:41.770Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:44.822Z: Checking permissions granted to controller Service Account.
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:48.716Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.231Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.271Z: Expanding GroupByKey operations into optimizable parts.
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.298Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.375Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.403Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.438Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.467Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.787Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 6:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:45:49.854Z: Starting 5 workers in us-central1-f...
Apr 12, 2020 6:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:46:14.016Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 12, 2020 6:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T18:46:18.749Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2020 6:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:46:34.914Z: Workers have started successfully.
Apr 12, 2020 6:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:46:34.938Z: Workers have started successfully.
Apr 12, 2020 6:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:47:09.389Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 6:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:47:09.579Z: Cleaning up.
Apr 12, 2020 6:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:47:09.658Z: Stopping worker pool...
Apr 12, 2020 6:49:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:49:07.589Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 12, 2020 6:49:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T18:49:07.626Z: Worker pool stopped.
Apr 12, 2020 6:49:13 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-12_11_45_41-10608369291339296041 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 05989cf6-d4bb-4310-b21f-97d0c4450a1b and timestamp: 2020-04-12T18:49:13.616000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.952
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 50.739 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 56s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7azxi4xgnvqkw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #375
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/375/display/redirect>
Changes:
------------------------------------------
[...truncated 282.79 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 12, 2020 12:52:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 12, 2020 12:52:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 12, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 12, 2020 12:52:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-wfCnI-a0m4B5VIeCtFR5ng.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-h5hO2eyLI0sTq9ZATVSRvw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-xGJSdAdiB5uJ-FvyZw7_Dw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-CRSjC6fzG2prPv6S-GZozA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-4S61-cWdIIe3c8vXr-BDFw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-8nlRfLuKmEIbX_g_r7__Tg.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-LNHMz22hhfba9HhAOBv7Rg.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-h4xc_HufcuQ-P9ENdPeb8w.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-KytRHZfOpgGiyxgCZPzixw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-n6NdPiSK3qXfv3cI1n2PpA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-mCyM5FvInh1NZy1nuUtd4Q.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-dXidowdt0DrqPJgLPt37sw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Zz-eoRHrvfxSsquiYlEbzQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-RujhXuYbaFtjMpzSmAx4zQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-8ojcuMVkf_M4kh2_ZZOuFA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-VcYBmrt0nUeeBiecmBxwdA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-_tpiqPMhG98r_BknVQBTVQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-NEcgfnwVjormrDYKR9phmA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-THtJYwLwy5-W5j0zJBkY3g.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-B0KV0G6OnCjx3GBs7pdHmQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-YjRwn0IwtX90HxsC_IH2kg.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-OmWEzNBd7O41BvYBQrxwlQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-E-YnOibr6M6cgxa6O2H4tA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-vMdtqPjGozfNyVoyEM_NYQ.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-EZ8x-HV8GHcqPoroN0fcIw.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-G2Dnwf4kEc0Joq5hh1mmJA.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-csGA1yb4PG4DZhF_srd9Eg.jar
Apr 12, 2020 12:52:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-csGA1yb4PG4DZhF_srd9Eg.jar
Apr 12, 2020 12:52:43 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-csGA1yb4PG4DZhF_srd9Eg.jar&uploadType=resumable&upload_id=AEnB2UrmH2mzuUzmWB5vKAxc-On5T27uPdxWvbNcEWVy5HhKa8aDhCZ8nJK0vVK4k7R5dEGzrpF2E6J1jufrtaVIrJBaqkeRfg.
Apr 12, 2020 12:52:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 12, 2020 12:52:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-csGA1yb4PG4DZhF_srd9Eg.jar
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 7 seconds
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 12, 2020 12:52:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash XoeeqwPLeF72k_Vs8x7Llw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XoeeqwPLeF72k_Vs8x7Llw.pb
Apr 12, 2020 12:52:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 12, 2020 12:52:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-12_05_52_50-5143777342498085172?project=apache-beam-testing
Apr 12, 2020 12:52:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-12_05_52_50-5143777342498085172
Apr 12, 2020 12:52:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-12_05_52_50-5143777342498085172
Apr 12, 2020 12:52:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T12:52:50.238Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 12, 2020 12:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:53.423Z: Checking permissions granted to controller Service Account.
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:56.927Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.514Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.549Z: Expanding GroupByKey operations into optimizable parts.
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.577Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.656Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.687Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.713Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:57.743Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:58.283Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:52:58.361Z: Starting 5 workers in us-central1-f...
Apr 12, 2020 12:53:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:53:21.932Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 12, 2020 12:53:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T12:53:31.112Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2020 12:53:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:53:39.793Z: Workers have started successfully.
Apr 12, 2020 12:53:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:53:39.833Z: Workers have started successfully.
Apr 12, 2020 12:54:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:54:16.092Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 12:54:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:54:16.291Z: Cleaning up.
Apr 12, 2020 12:54:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:54:16.366Z: Stopping worker pool...
Apr 12, 2020 12:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:55:41.778Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 12, 2020 12:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T12:55:41.814Z: Worker pool stopped.
Apr 12, 2020 12:55:48 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-12_05_52_50-5143777342498085172 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e15d4dea-7ac0-44d4-b8ce-425147f3c4a3 and timestamp: 2020-04-12T12:55:48.067000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.659
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.04 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 16.618 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 20s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/sf3pbroes4lsm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #374
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/374/display/redirect>
Changes:
------------------------------------------
[...truncated 282.87 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 12, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 12, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 6:45:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 12, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 12, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 12, 2020 6:45:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 12, 2020 6:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-H7PASmJPU8RUUFcwgOOwFw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-q9GYKvKpAN-Ss1-JYTvnFA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-5c4UHx81nRGWJQZGFvDdFA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-i8aqAZQ-AuVNMjAmCftOdg.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TEvJsIomth61fUzHzTuOPw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-LuTndZXcB85cCsncBzy0tA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-pI1Rifw2P-nWLgPJnBv61g.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-X5mDTnjnmDhSzxN_Az9UWw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-lsuT1kY_Hr0tGOllJ3NS9w.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-kngLD6DLj24lfMQlPEV_Vw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-XNS2pgSnIEMSNo_tGgxlzQ.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-2eNCAICb0lBqIQWHFcQoWg.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-wqzOfy4iMqDEr3UVL6HftA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-ShzKW3MT-G2Jwy6uMCIpiA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-xrj-53J3RdsLdnfMo4V2BA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-OqHLeLyGZcFOT8QTTvMXag.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4DDgMC039MkkjLWTwL3vvA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-aBTY807xKzXDtbQS_1K4Ig.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-BkfeRZyaMAFvGyai39DreQ.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-EmvFtOVTEVI5qb7e2HeNZg.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-qOl0CJ4HCXRN4OYcqu-CAw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-7JDjiZ9eudOvasb0by8naw.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4DDgMC039MkkjLWTwL3vvA.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-DDLzKLV1e44GaGdWcNQKGg.jar
Apr 12, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-lL4nkegG-2zQzCbGo44i0A.jar
Apr 12, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-nFtA4Oz0cjB1axNGLnF2Aw.jar
Apr 12, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-A3gS3DasxIavuo4NpA_P2Q.jar
Apr 12, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-JIqCyvuIHQaYNKt_3kgWjw.jar
Apr 12, 2020 6:45:19 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4DDgMC039MkkjLWTwL3vvA.jar&uploadType=resumable&upload_id=AEnB2UqWpPIxO9wwGaOk3GIl2e7RnLY8VWMJ9zdj9j5kZjTBJNWirAhFW4yBz-THapuloFahy-mwqTIH8d9ExkiV6z4Xj_bo1PLL1ZVB7G-FurNpZBlleTk.
Apr 12, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 12, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-4DDgMC039MkkjLWTwL3vvA.jar
Apr 12, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 8 seconds
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash BHpz2lzxQfDucmfGfoQBWQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BHpz2lzxQfDucmfGfoQBWQ.pb
Apr 12, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 12, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-11_23_45_26-16158396268199047144?project=apache-beam-testing
Apr 12, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-11_23_45_26-16158396268199047144
Apr 12, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-11_23_45_26-16158396268199047144
Apr 12, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T06:45:26.944Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 12, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:30.028Z: Checking permissions granted to controller Service Account.
Apr 12, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:33.260Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:33.857Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:33.896Z: Expanding GroupByKey operations into optimizable parts.
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:33.921Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:33.989Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:34.021Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:34.044Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:34.070Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:34.396Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:45:34.478Z: Starting 5 workers in us-central1-c...
Apr 12, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T06:45:51.381Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:46:03.341Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 12, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:46:26.004Z: Workers have started successfully.
Apr 12, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:46:26.035Z: Workers have started successfully.
Apr 12, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:47:07.736Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:47:07.942Z: Cleaning up.
Apr 12, 2020 6:47:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:47:08.025Z: Stopping worker pool...
Apr 12, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:48:19.531Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 12, 2020 6:48:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T06:48:19.568Z: Worker pool stopped.
Apr 12, 2020 6:48:26 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-11_23_45_26-16158396268199047144 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b7245539-67a4-4de7-ac9a-89675662ee8e and timestamp: 2020-04-12T06:48:26.322000000Z:
Metric: Value:
read_time 21.662
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 19.482 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 8s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/xkxh2qwp3woag
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #373
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/373/display/redirect>
Changes:
------------------------------------------
[...truncated 282.90 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 12, 2020 12:45:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 12, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 12, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-eCVFza2SKJEIjx0nHvbH2w.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-I8xgs91bSGorIu3dtnnOig.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-ByROGGneyQKdRfVTNJ7s6g.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-kAJxzHTzw_x7tCgdUoifEQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-V4T6enAvfmHBBEHEHHp_2g.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-y73GnEMVGuEjoa-fQJUNbg.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Fs_NpeM65wQQQazvaadGhQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-w1fhdvF4u-BoT1RZHy0d7A.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Z7tJdmnXyQ7SrzdEIapdSQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-regFH6ORj0_e0wwn8PEClQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-zOVXz2U_-eR37uvBQ0YVqQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-fYqyTbz9WZtry6EM5D5-Xw.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-nvejNdDXqDtMZdjSKsMwpQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-cdCF1V4hVwU468CIxAnp0g.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-NaAWa3ar7htjBiKNbJ7AAA.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-PQb55Gv1sYSZxa1Me0Z2kA.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-5h681r10q-L9vRlvONh9gw.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-0XelWYAJaHlsaDnTMpaZQA.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-YFXFq29rpR-CXnlBXC_GFQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-fpYpgxybXmDEkBm3dN7cRQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-p3xeL1VbkI1St_trafvD2A.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-ft2WnzyMDjIo0ApNwoWNNw.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-ekWhscYLZBllHtvVxPsZ6w.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-nKQd9i7sK67Sh-i2pRn0Tg.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-ut4xXc20xFjmwZny1IyEZA.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3B1RfME5QzfNHcntGlo2aQ.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gL-OjLrLGddNMG_lBliWdg.jar
Apr 12, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3B1RfME5QzfNHcntGlo2aQ.jar
Apr 12, 2020 12:45:31 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3B1RfME5QzfNHcntGlo2aQ.jar&uploadType=resumable&upload_id=AEnB2Ur1wf61H0EV7gXVgnVLIeP-vz4wL7vhsMQcDefXi6qy5g4sypgwLmHFnBe8MiXyu3mag-KjMm3BjKXfphy6WiE6bxfwXGhpWEIXbcqMPL8f6T1fQTc.
Apr 12, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 12, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-3B1RfME5QzfNHcntGlo2aQ.jar
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 5 seconds
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 12, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 12, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 1ccWETwGoqcuD9eLVRJs5Q> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1ccWETwGoqcuD9eLVRJs5Q.pb
Apr 12, 2020 12:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 12, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-11_17_45_36-2198280080032592481?project=apache-beam-testing
Apr 12, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-11_17_45_36-2198280080032592481
Apr 12, 2020 12:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-11_17_45_36-2198280080032592481
Apr 12, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T00:45:36.485Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 12, 2020 12:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:39.870Z: Checking permissions granted to controller Service Account.
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:43.916Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.440Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.484Z: Expanding GroupByKey operations into optimizable parts.
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.537Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.613Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.647Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.679Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:44.716Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:45.147Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 12:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:45:45.239Z: Starting 5 workers in us-central1-f...
Apr 12, 2020 12:46:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-12T00:46:08.186Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2020 12:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:46:11.380Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 12, 2020 12:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:46:11.412Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 12, 2020 12:46:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:46:16.876Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 12, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:46:29.693Z: Workers have started successfully.
Apr 12, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:46:29.731Z: Workers have started successfully.
Apr 12, 2020 12:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:47:03.105Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 12, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:47:03.288Z: Cleaning up.
Apr 12, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:47:03.378Z: Stopping worker pool...
Apr 12, 2020 12:48:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:48:55.110Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 12, 2020 12:48:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-12T00:48:55.156Z: Worker pool stopped.
Apr 12, 2020 12:49:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-11_17_45_36-2198280080032592481 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9dfe387d-e384-4330-98e1-78f759429043 and timestamp: 2020-04-12T00:49:00.261000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.559
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 40.08 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 43s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ohmcope2gtyvu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #372
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/372/display/redirect>
Changes:
------------------------------------------
[...truncated 282.10 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 11, 2020 6:48:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 11, 2020 6:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 11, 2020 6:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-qKvYPIeD6fjGrmLLR0RNfQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-iB7DoAO8RxWzePw3LlqweQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-EGDguNw2mJJwV4207Biavw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-Nzxh2mTZ3sHgAqJPLQt21A.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-mk2s7gtYo2Qb--oxjXSNPQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-RQNlBjSEivP9ku8X3nZVPg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-hiMiubMnvRxuff99yVuDUw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-B7bRO_k41gZ6UK4vk6JvtA.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-wLhtOzg6TefMd9RugO9P-Q.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-XSdBHwqX4sfAUAWCV_nYQg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-hsjkLpOjLhOBAEFNcrcRFA.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-H31N7BTx6j8L9bu1InFtaQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-pHKQSWI4zxksH2hU5EridQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-dgOpQjRgKN3Q8r4JcB8Otw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-RuVqe6G2Thj_UIh32VkLNg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-umeatttl3HDvE0xLAe94yw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-6lWtPm3EwY3tocVL3YYWeQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-8sN85Dp8AWEhzRfzIJcmpg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-us3grc_iyVFE0pPT__MoiA.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-9e-wJwlwPmzKB55aOCVv1g.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-WRXU_kjtyoG5ZNitSASe6A.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oc1cWql8Fnyq9jGMeIOuDw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-zR96Q5wwn0KbaVtT_MwK0Q.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests--2h4KPgiDFfYJp4mcboROg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-iMM-X06PON28UcRaqvfDUQ.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oc1cWql8Fnyq9jGMeIOuDw.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-OdCT2-naz7-17GSrWP0BOg.jar
Apr 11, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-2GyUBb83v0EwAJoYL0niig.jar
Apr 11, 2020 6:48:20 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oc1cWql8Fnyq9jGMeIOuDw.jar&uploadType=resumable&upload_id=AEnB2UpYIR6V_gbfuYaDMDqnd09ASR73UX3K2G1RDiuhrBY2Y1kwy19OjPq8DBX4TCZC889HWM81hlXQg9Ij8mwsYWVuFUIsr7vhCAgUHulYwpWgyCW5Q9s.
Apr 11, 2020 6:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 11, 2020 6:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oc1cWql8Fnyq9jGMeIOuDw.jar
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 11, 2020 6:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash zROqKQLFT-fWXJ2NOFxATA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-zROqKQLFT-fWXJ2NOFxATA.pb
Apr 11, 2020 6:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 11, 2020 6:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-11_11_48_26-1682753476350738842?project=apache-beam-testing
Apr 11, 2020 6:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-11_11_48_26-1682753476350738842
Apr 11, 2020 6:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-11_11_48_26-1682753476350738842
Apr 11, 2020 6:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T18:48:26.288Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 11, 2020 6:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:29.456Z: Checking permissions granted to controller Service Account.
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:32.438Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:32.950Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:32.983Z: Expanding GroupByKey operations into optimizable parts.
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.011Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.090Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.121Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 11, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.153Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 11, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.192Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 11, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.532Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 6:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:48:33.614Z: Starting 5 workers in us-central1-f...
Apr 11, 2020 6:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T18:48:59.954Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 11, 2020 6:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:03.581Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 6:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:03.608Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 11, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:08.993Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 6:49:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:23.061Z: Workers have started successfully.
Apr 11, 2020 6:49:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:23.092Z: Workers have started successfully.
Apr 11, 2020 6:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:54.187Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 6:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:54.342Z: Cleaning up.
Apr 11, 2020 6:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:49:54.415Z: Stopping worker pool...
Apr 11, 2020 6:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:51:54.860Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 11, 2020 6:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T18:51:54.895Z: Worker pool stopped.
Apr 11, 2020 6:52:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-11_11_48_26-1682753476350738842 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): cb054288-d5ba-4fb8-85bd-43de9c04c501 and timestamp: 2020-04-11T18:52:01.468000000Z:
Metric: Value:
read_time 14.383
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 51.833 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 51s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/d3bfszsr67try
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #371
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/371/display/redirect>
Changes:
------------------------------------------
[...truncated 283.12 KB...]
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 11, 2020 12:48:42 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 11, 2020 12:48:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 11, 2020 12:48:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-5bEuawLhsvo1aLJPb4r0ZQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-2IzZhn5xj6weKjIk7gAW_w.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-PUxgzhF-3sRlKWnSttfVgg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-stWOtfA0egwBdRy3h_M0bA.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-mf1PGisjv4dDYnYfHDbdxg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-MwZoIrNfenSfg3rVGbZz3g.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests--bGbOP_DW17gM-dyfbCzdQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-98rEt3-D59fbWzbYgTeXcg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-QNhO8OecI5PxCVay73xmrQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-dCgS_Z1Xafv0BPVB0UyqLQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-7JGaCeam-nL_O1ZMzUAxyg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-UQ0_cnUWEupheKVAc0vZBw.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-75AWGs-BqQdWi-Ij0bVGNQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-mqbWqDYafdem_nh3f3ArTg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-Ag5lQ0pXTSVxQ4qqQ_a1Bw.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-2-Zj-2DF-Nij4ibOfIWVJg.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-z5Puu4nTSbn8Wh795p9yGA.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-QCfaMiL6tUyrLXUormcJ1A.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-SEWjFMcxGvv1PiGW53egkQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-wdOhiZ9kKQgOx0PkS3LaZA.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-bGs0WV9eU58UjWvK_3xylQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-8adHjPqwwgZGLBUPxcmEfA.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-QwHOimP6-KGRKZ9K75V4Sw.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-z11RjhAgL9uF0hPho9jnAQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-csv-1.8-kBnZnWBy9I_ZsfkJ8MRdJA.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-lXObkFJ0t3EHjk7phhxpcQ.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-B2Z_PGVHNfFBQ8og-_ua1A.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-B2Z_PGVHNfFBQ8og-_ua1A.jar
Apr 11, 2020 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-UvaRZPi54bA32rfoJlePWA.jar
Apr 11, 2020 12:48:45 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-B2Z_PGVHNfFBQ8og-_ua1A.jar&uploadType=resumable&upload_id=AEnB2UqjtdUHvW0bA8g_rWTu_GNSQ6IIKiEfQgZ_xRp5x9BOjCBXOAi083JNd4Jn5I-wlKnvzh18fuLL_fFbVriUKw23qOMo85cAiCUNJt96HDgam0pVZKc.
Apr 11, 2020 12:48:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 11, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-B2Z_PGVHNfFBQ8og-_ua1A.jar
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 177 files cached, 29 files newly uploaded in 6 seconds
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 11, 2020 12:48:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 66otiHelIC_5qGDYh63_Kw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-66otiHelIC_5qGDYh63_Kw.pb
Apr 11, 2020 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 11, 2020 12:48:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-11_05_48_51-2485652238096942838?project=apache-beam-testing
Apr 11, 2020 12:48:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-11_05_48_51-2485652238096942838
Apr 11, 2020 12:48:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-11_05_48_51-2485652238096942838
Apr 11, 2020 12:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T12:48:51.428Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 11, 2020 12:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:54.689Z: Checking permissions granted to controller Service Account.
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:58.414Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.011Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.041Z: Expanding GroupByKey operations into optimizable parts.
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.071Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.135Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.161Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.186Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.222Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 11, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.508Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 12:49:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:48:59.564Z: Starting 5 workers in us-central1-a...
Apr 11, 2020 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T12:49:18.864Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 11, 2020 12:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:49:29.754Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 12:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:49:29.788Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 11, 2020 12:49:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:49:35.147Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:49:49.446Z: Workers have started successfully.
Apr 11, 2020 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:49:49.485Z: Workers have started successfully.
Apr 11, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:50:29.498Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:50:29.669Z: Cleaning up.
Apr 11, 2020 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:50:29.728Z: Stopping worker pool...
Apr 11, 2020 12:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:51:51.562Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 11, 2020 12:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T12:51:51.598Z: Worker pool stopped.
Apr 11, 2020 12:51:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-11_05_48_51-2485652238096942838 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 5cf68b34-d042-47e6-81dd-aaa687456837 and timestamp: 2020-04-11T12:51:58.004000000Z:
Metric: Value:
read_time 24.023
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 23.908 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 27s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ws5zgx6eb5oty
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #370
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/370/display/redirect>
Changes:
------------------------------------------
[...truncated 282.75 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 11, 2020 6:45:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-YF1ADR7bH8_ztYdgyUhxrw.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-aeERtjWSVqdOIQiEo0u1vw.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-RkjtCpoERwmhUa059Pcp3A.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-wE_eZlusC9B3LMZjfQO0YQ.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-S4vlxiZuHdkSjkQinvd1IA.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-2NTAgkk_lqcY3G5I641wIA.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-op9MPt54TXQc9_M5puuXLg.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-cmYhzvrEjIQzqzZZrZcXmA.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-Nlya6d5Wtqvo9OHHGJkCjw.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-Ut1y-Yrwt_Sth2iM16_rmg.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-b--5AEZl2WhO3HkHsbfL7w.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-ygIAMoqFWf-epcSlHd7KtA.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-7XqkbSxtBggXrf5veBjkUw.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-Cet-YgYczIxGCQFmUUyV0A.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-EZ0QxKe_0Tsnfda_epWz8w.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-WxLCfbcpUpWz7iFJ7o_XDQ.jar
Apr 11, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-s2xI5lZXX0Zb9jULntywnA.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-N2gBWrsbWcfNL9Lu0j6zfA.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-4joh4HBZil02TnjFBmO_0Q.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-COQwrgTnPMaJulm7cS2IcQ.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-jZ8O5Oq-gTmMVjdBzFgMJg.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-BuwJb3L6A-JoJq2gqcULxA.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-WKunpy0ceS3EaRJaOLzBcw.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-r4eqpfR4K1l4arGiDgL1nw.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mBp3BUgnZh_YuhZ1M6lJ0g.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mBp3BUgnZh_YuhZ1M6lJ0g.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-t3Hs1fGBXaoPGVTHyo7dmA.jar
Apr 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-dWj89sJj1abNoD-as7J_Sw.jar
Apr 11, 2020 6:45:26 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mBp3BUgnZh_YuhZ1M6lJ0g.jar&uploadType=resumable&upload_id=AEnB2UqqcA8Bdqt9mB9eL75-E7R41664U1QHEn1pP_ADpX1m0CkCvuoUBNuxaZC0KAp9I1KslqwkGgQfy6ht-Z2jHSsobMPBKy4vdtkI3inH-7PFPMDfhvQ.
Apr 11, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 11, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-mBp3BUgnZh_YuhZ1M6lJ0g.jar
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 11, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash ZbZyoezoATMzpMXQrpbD-w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZbZyoezoATMzpMXQrpbD-w.pb
Apr 11, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 11, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-10_23_45_31-14204541363696671739?project=apache-beam-testing
Apr 11, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-10_23_45_31-14204541363696671739
Apr 11, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-10_23_45_31-14204541363696671739
Apr 11, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T06:45:31.611Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 11, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:35.214Z: Checking permissions granted to controller Service Account.
Apr 11, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:38.670Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 11, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.309Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.356Z: Expanding GroupByKey operations into optimizable parts.
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.393Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.487Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.528Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.563Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.601Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:39.985Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:45:40.069Z: Starting 5 workers in us-central1-f...
Apr 11, 2020 6:46:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:46:04.424Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 6:46:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:46:04.460Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 11, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:46:09.878Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T06:46:11.791Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 11, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:46:28.115Z: Workers have started successfully.
Apr 11, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:46:28.147Z: Workers have started successfully.
Apr 11, 2020 6:47:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:47:00.839Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 6:47:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:47:01.022Z: Cleaning up.
Apr 11, 2020 6:47:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:47:01.108Z: Stopping worker pool...
Apr 11, 2020 6:49:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:49:02.501Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 11, 2020 6:49:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T06:49:02.550Z: Worker pool stopped.
Apr 11, 2020 6:49:08 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-10_23_45_31-14204541363696671739 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f094155f-4f59-4878-b8c3-5cb050e25143 and timestamp: 2020-04-11T06:49:09.005000000Z:
Metric: Value:
read_time 15.003
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 53.819 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7qsw3zlpyz6cu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #369
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/369/display/redirect?page=changes>
Changes:
[robertwb] Attempt to stage resources via new API in portable runner.
[ankurgoenka] [BEAM-9735] Adding Always trigger and using it in Reshuffle
[pabloem] Fix from_container_image call
[ankurgoenka] TOIL: Update Unified worker image
[boyuanz] [BEAM-9562] Update Element.timer, Element.Timer to Element.timers and
[github] [BEAM-9443] support direct_num_workers=0 (#11372)
------------------------------------------
[...truncated 284.16 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 11, 2020 1:14:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 11, 2020 1:14:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 11, 2020 1:14:18 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-yi2Kum1BTgrboWqOhb-R0Q.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-coVfP4Jl_4xJPdF7L3mVyg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-CfoDFNkEWFy4rUaPd_f-ow.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-MaB3tLVnNNirspR_6lanbw.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-5tFS39JmknEmBltS37EFSg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-lblGEAX3IsAfR6t4BZCa2g.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-_7sdozTM5B5z6SYPBPY5qQ.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-IvqjJJEgCSRd5beiPn7KcA.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT--wfsi1bkIfM54ccr8yB9tg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-2Nhoo7mevBIPhBS1aQSMmA.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-heWCTee6Kn7ImqgqJZOakg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-bgSwboWOP2oNSPGQyyTfWw.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-wksq1r2E0On7nL262HT8ow.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-WG7n_zvPfI-i6DSC6MVsyA.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-xNRR-nEhWY42B_zFfRAO3g.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bwCMXaIvAALXIJLz8mTidA.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-hDBPvUfNL1MN3ksukyjkjw.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-wSMjkj9aYghYq2fWyBln4g.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-zhVnxzbwcFP6qVN4Qssg5A.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-MHf0eIzvuj95uJ9DqdAcPg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-R2kP-GaHmT2pDjkq-XAoTg.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-GEM18ddH4XvtW7nr1wu9bQ.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-8UUV7ZKHJx43CN1x4rZAaA.jar
Apr 11, 2020 1:14:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-gPHJ-67FPfKOWysqewhKAA.jar
Apr 11, 2020 1:14:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VzFgrv3Bz659UytfZ7YqEQ.jar
Apr 11, 2020 1:14:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-AibjA3NyPHxxkwvuCR6Lew.jar
Apr 11, 2020 1:14:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-orQQtTUYnUMz33IEeY9QmQ.jar
Apr 11, 2020 1:14:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VzFgrv3Bz659UytfZ7YqEQ.jar
Apr 11, 2020 1:14:21 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VzFgrv3Bz659UytfZ7YqEQ.jar&uploadType=resumable&upload_id=AEnB2Uq86JSpRSTlLiualmqbieEt-n-FD2V-vt-LblMcZI0p9Mzw5S6SPHfFFdfmgq5psLeqT7pWuTI5m4MVULrPh-qyzqMIvz6_xX4lEXSobSG7HhC-0k0.
Apr 11, 2020 1:14:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 11, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-VzFgrv3Bz659UytfZ7YqEQ.jar
Apr 11, 2020 1:14:28 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 10 seconds
Apr 11, 2020 1:14:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash 8rlh3YmQNpTv48IKwxumgg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8rlh3YmQNpTv48IKwxumgg.pb
Apr 11, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 11, 2020 1:14:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-10_18_14_29-5850018895119724717?project=apache-beam-testing
Apr 11, 2020 1:14:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-10_18_14_29-5850018895119724717
Apr 11, 2020 1:14:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-10_18_14_29-5850018895119724717
Apr 11, 2020 1:14:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T01:14:29.903Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 11, 2020 1:14:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:33.932Z: Checking permissions granted to controller Service Account.
Apr 11, 2020 1:14:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:38.273Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:38.970Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.005Z: Expanding GroupByKey operations into optimizable parts.
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.101Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.198Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.235Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.272Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.317Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.825Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 1:14:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:14:39.909Z: Starting 5 workers in us-central1-f...
Apr 11, 2020 1:14:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-11T01:14:47.670Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 11, 2020 1:15:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:15:08.785Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 1:15:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:15:08.812Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 11, 2020 1:15:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:15:14.306Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 11, 2020 1:15:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:15:31.211Z: Workers have started successfully.
Apr 11, 2020 1:15:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:15:31.246Z: Workers have started successfully.
Apr 11, 2020 1:16:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:16:05.850Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 11, 2020 1:16:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:16:06.084Z: Cleaning up.
Apr 11, 2020 1:16:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:16:06.197Z: Stopping worker pool...
Apr 11, 2020 1:17:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:17:45.439Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 11, 2020 1:17:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-11T01:17:45.495Z: Worker pool stopped.
Apr 11, 2020 1:17:52 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-10_18_14_29-5850018895119724717 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 717cc250-3004-40eb-a481-f2dc99df87c8 and timestamp: 2020-04-11T01:17:52.058000000Z:
Metric: Value:
read_time 16.59
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 43.262 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 3s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/knqkkiovkbcvm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #368
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/368/display/redirect?page=changes>
Changes:
[veblush] Upgrades gcsio to 2.1.2
[samuelw] [BEAM-9651] Prevent StreamPool and stream initialization livelock
[michal.walenia] [BEAM-9734] Revert #11122
[github] Add --region to changelog
------------------------------------------
[...truncated 302.17 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 10, 2020 6:57:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 6:57:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 10, 2020 6:57:12 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 6:57:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 10, 2020 6:57:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 10, 2020 6:57:12 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 10, 2020 6:57:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 10, 2020 6:57:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-lMH0Gu8gxuYVYFDXvzjcbw.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-LZMSCsRMQwfgSj2GkB4ulg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-F0LMzZPGHSzpH4PtHPR4Gg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-dp3r209HzPNqI-zHwAFTkw.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-LeXZuWvGsEh4sK87-XqsoA.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-oKgfaEbDJP7zI8HWV76y5A.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-y4PNHEl5ky7zqbBusZgatg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-zctm8WRXQtJQTSC8mtR2hA.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-quzR7Q7qdHGEYcsAG95f9A.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-au_f60-lr4ZVtje0a0Oldg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-ENskKCsxylq-VwzGqJklOQ.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-H3m49eGpBrgfkyBrfHUlOg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-_kqNi0QfSTTDt2Omj3iejw.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-icXP2xuZDGhfqttrmstdKg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-Ml9zxxj59PslqvDCdJpwzA.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-AFDpRexgVSEKeZl97v59Yg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-DzuIMkRy1P2Y2w2z_j0KtA.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-aB4IzcF8lVQnMmwWkHqB3A.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-SU90_UDo91p9tW9kxazHQg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-9-3rIr2oc-0NNNcHhEBRZw.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-KLJNziwV-kqeKEmgo43kEg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-RNbuREokeER2ZHZL4UR8uA.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-ot0cLQV5tJInXEnlAR_LLg.jar
Apr 10, 2020 6:57:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-sHV6GsyeorMMEFAd9q4IyQ.jar
Apr 10, 2020 6:57:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-11vPDGzZ4XZdvuikBmPN4Q.jar
Apr 10, 2020 6:57:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-24L4WPc4o11mskezRm-0wA.jar
Apr 10, 2020 6:57:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-24L4WPc4o11mskezRm-0wA.jar
Apr 10, 2020 6:57:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-ZLfsQt3IEOOf1mNHXM2ouQ.jar
Apr 10, 2020 6:57:15 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-24L4WPc4o11mskezRm-0wA.jar&uploadType=resumable&upload_id=AEnB2UrmYdjbf7xdlJd7jjA5NIVnhSGcgiLCQTXIb2s45DzYzu6OnSqFEZfQdLInrFbRxyWB5b5x2zdIMwc4F1ZOByr_q23AdPQf5MA5uhNO6uED3DiZr3w.
Apr 10, 2020 6:57:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:355)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:230)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:318)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 10, 2020 6:57:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-24L4WPc4o11mskezRm-0wA.jar
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 9 seconds
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 10, 2020 6:57:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12102 bytes, hash zL9dVF9jnGsnpzmhGRk3Iw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-zL9dVF9jnGsnpzmhGRk3Iw.pb
Apr 10, 2020 6:57:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 10, 2020 6:57:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-10_11_57_23-4901453625383912537?project=apache-beam-testing
Apr 10, 2020 6:57:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-10_11_57_23-4901453625383912537
Apr 10, 2020 6:57:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-10_11_57_23-4901453625383912537
Apr 10, 2020 6:57:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T18:57:23.369Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 10, 2020 6:57:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:26.922Z: Checking permissions granted to controller Service Account.
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:30.764Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.397Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.441Z: Expanding GroupByKey operations into optimizable parts.
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.483Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.577Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.616Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.655Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:31.690Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:32.182Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 6:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:32.271Z: Starting 5 workers in us-central1-f...
Apr 10, 2020 6:57:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T18:57:47.707Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 10, 2020 6:57:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:57:56.974Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 10, 2020 6:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:58:16.343Z: Workers have started successfully.
Apr 10, 2020 6:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:58:16.369Z: Workers have started successfully.
Apr 10, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:58:48.946Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:58:49.119Z: Cleaning up.
Apr 10, 2020 6:58:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T18:58:49.193Z: Stopping worker pool...
Apr 10, 2020 7:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T19:00:42.805Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 10, 2020 7:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T19:00:42.853Z: Worker pool stopped.
Apr 10, 2020 7:00:48 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-10_11_57_23-4901453625383912537 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2e2dc48d-475b-4dc7-a72b-2d7f8f5e43e4 and timestamp: 2020-04-10T19:00:48.892000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.615
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.067 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 45.932 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 6s
101 actionable tasks: 100 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/mdfu7k52tsmxy
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #367
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/367/display/redirect>
Changes:
------------------------------------------
[...truncated 282.84 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 10, 2020 12:47:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 12:47:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 10, 2020 12:47:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 12:47:10 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 10, 2020 12:47:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 10, 2020 12:47:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 10, 2020 12:47:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 10, 2020 12:47:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-SWDL6vNXbGouk4unnSD-7Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-r1DDEF10qOA4UxEpCYsAKw.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests-zIC1KlHK3gSl8Y2KpKKL9Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-R9x0E1lKDI4ArVigGbOi9w.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-bb9Gpm64fd6JZCkQIXvSAA.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-d_a_9k4cLNpvR-n47YPREw.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TWqme_S7JEO7uIADq3akmQ.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT--4IcCyEGuGG7w7fH1aV_Gg.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-OOq-CY65krubc3KKwckWcQ.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-lR6QP0UzjzXbj1vN72yBKg.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-EsmTOA64onQB8ZYxp6PU8Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-t2ijwDq7nGI8IMwOHIi8Bg.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-M3IkK8rVqUwFquuWfqFfoQ.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-_wcecxdiP9S6IPF1TpG29g.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-F-ykG3zrh8z8-ArPRMPC8Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-ipVO81vQbTqMvh4KwH1F8A.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-DLnOkw5RU43KRUX-wRmBjg.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-5VTgg0SEoUMjZzvIDl-rjQ.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-rdhUJT0gAOH6EuDYopTawg.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT--CUJRzFAuSP9Yidb90rOWQ.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-6dXipkVVUnKFRq8P2Qux4Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZQAs1jtL1cs2yAWbuVEL8g.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-TQEhtBRMlyb4P2vsskGx5Q.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-v1YxsOt85Fot72LEc4X_bw.jar
Apr 10, 2020 12:47:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-Dte1lg4cBrHxjfkOOcNkew.jar
Apr 10, 2020 12:47:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZQAs1jtL1cs2yAWbuVEL8g.jar
Apr 10, 2020 12:47:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-zoQrc7QHtEfO8-ze7lYTdA.jar
Apr 10, 2020 12:47:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-yX2mjydBGsRg7EvEq_ka8Q.jar
Apr 10, 2020 12:47:14 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZQAs1jtL1cs2yAWbuVEL8g.jar&uploadType=resumable&upload_id=AEnB2UohJ3xLVvl-s4lDf5drpA-l0_8oKA2f3hegzQrA8HtbMk1SzqiLcBuja9mrsgKD7ml_2XTkVQ8uS1WGXPAgxpV8fr-nEwFFBQ1R9r59NGOctXoiGS0.
Apr 10, 2020 12:47:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:309)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:215)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:272)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 10, 2020 12:47:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-ZQAs1jtL1cs2yAWbuVEL8g.jar
Apr 10, 2020 12:47:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash PcZemeZSVxiaJFTRQqU0kA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PcZemeZSVxiaJFTRQqU0kA.pb
Apr 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 10, 2020 12:47:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-10_05_47_19-10703226236626866157?project=apache-beam-testing
Apr 10, 2020 12:47:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-10_05_47_19-10703226236626866157
Apr 10, 2020 12:47:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-10_05_47_19-10703226236626866157
Apr 10, 2020 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T12:47:20.043Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 10, 2020 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:23.444Z: Checking permissions granted to controller Service Account.
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:27.634Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.190Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.236Z: Expanding GroupByKey operations into optimizable parts.
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.274Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.370Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.457Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.489Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.531Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:28.997Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:29.089Z: Starting 5 workers in us-central1-f...
Apr 10, 2020 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T12:47:41.800Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 10, 2020 12:47:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:47:54.707Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 10, 2020 12:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:48:10.909Z: Workers have started successfully.
Apr 10, 2020 12:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:48:10.946Z: Workers have started successfully.
Apr 10, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:48:48.137Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:48:48.358Z: Cleaning up.
Apr 10, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:48:48.452Z: Stopping worker pool...
Apr 10, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:50:58.946Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 10, 2020 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T12:50:58.996Z: Worker pool stopped.
Apr 10, 2020 12:51:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-10_05_47_19-10703226236626866157 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): bcaa52fa-f864-4fec-bde5-061a0fc472b0 and timestamp: 2020-04-10T12:51:04.921000000Z:
Metric: Value:
read_time 19.343
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 4 mins 2.306 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 4s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/iixxwst3rgjwi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #366
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/366/display/redirect?page=changes>
Changes:
[kcweaver] Moving to 2.22.0-SNAPSHOT on master branch.
------------------------------------------
[...truncated 285.40 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 10, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 10, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 10, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 10, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-cMEM7VThBcP3zp7Jv2oGkA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-tests-K4simAMjZT9xxH92JmoZag.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-btcdIja4g02TPvwvCPZ29g.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-tests-vX0LQ5QBIToOkaUw17l9qg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-d1PrmTzdWbAwKdCLHvdITA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-ix3Snst_0266cF1YUToHkA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-KAvnRWmhZ2e9OKelKA6OMg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.22.0-SNAPSHOT-aunXgQHfHdpWUZ4_55eF8A.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-fi_xd95Cwbah1B4XmpjI4w.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-UWSIlUTNN2icEpCUpc2iew.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-MJsc3Jy6S4xWzmeImkMXLg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.22.0-SNAPSHOT-tests--F13m3HnoSup4w-k3gU4nA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.22.0-SNAPSHOT-JvSXYAVqTITAuvj_eIt1vw.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-lbvA6OA7WNFTl7jffSXkbA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-tests-nYGOYK_2choIumi_xQkHqQ.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.22.0-SNAPSHOT-tests-IzKK2oYztUVA0poSTTCowg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.22.0-SNAPSHOT-tests-Knd0Asd5JtXSxJK2zzUKmA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-iBCNOGrLmA9CEh8yi1HZuA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-unshaded-njI19OOwd7dp1uGctRSdBQ.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.22.0-SNAPSHOT-QucS667Fq1cRck_JzSAVig.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-xsD-pm1NP_xyJH-FFJYkoA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.22.0-SNAPSHOT-fV3joOKuHsvg51ogznmXSA.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-avV4SOHjvRL1VEjrl-OBCg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.22.0-SNAPSHOT-JaKrUyYf29GFZMS3daVwpg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oa_VcPxrCOnlsI2WE8I_gg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-tests-wPndHX0AnemIl64trSX5Wg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oa_VcPxrCOnlsI2WE8I_gg.jar
Apr 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-tests-AZQJxITyXM4-pfX-HAa2NA.jar
Apr 10, 2020 6:45:46 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oa_VcPxrCOnlsI2WE8I_gg.jar&uploadType=resumable&upload_id=AEnB2Upoy8ukPxkn6u0CTdtNpfa1L8Q2i7camx9X0CfP-Ba4tXVvfyU3lJTPGg-uCuKi6HJb8cNfJdimy47pXPfnoWIyuVeO7A.
Apr 10, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:309)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:215)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:272)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 10, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.22.0-SNAPSHOT-oa_VcPxrCOnlsI2WE8I_gg.jar
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 6 seconds
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash bCq-wILVTHs3eos-tlPhFg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bCq-wILVTHs3eos-tlPhFg.pb
Apr 10, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-09_23_45_51-5540603260631683872?project=apache-beam-testing
Apr 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-09_23_45_51-5540603260631683872
Apr 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-09_23_45_51-5540603260631683872
Apr 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T06:45:51.290Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 10, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:54.565Z: Checking permissions granted to controller Service Account.
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:57.663Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.179Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.213Z: Expanding GroupByKey operations into optimizable parts.
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.235Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.309Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.338Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.358Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.386Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.670Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:45:58.731Z: Starting 5 workers in us-central1-c...
Apr 10, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:46:22.884Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 10, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T06:46:23.805Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 10, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:46:41.284Z: Workers have started successfully.
Apr 10, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:46:41.385Z: Workers have started successfully.
Apr 10, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:47:21.403Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:47:21.619Z: Cleaning up.
Apr 10, 2020 6:47:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:47:21.707Z: Stopping worker pool...
Apr 10, 2020 6:48:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:48:32.772Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 10, 2020 6:48:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T06:48:32.827Z: Worker pool stopped.
Apr 10, 2020 6:48:39 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-09_23_45_51-5540603260631683872 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8b387d62-83b5-4c89-8119-b620550c47d1 and timestamp: 2020-04-10T06:48:39.139000000Z:
Metric: Value:
fields_read 4375276.0
read_time 21.322
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 4.404 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 21s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/wmqwfpptgh7tg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #365
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/365/display/redirect?page=changes>
Changes:
[github] add missing bracket
[kcweaver] [BEAM-9726] [py] Make region optional for non-service Dataflow.
[kcweaver] [BEAM-9726] [java] Make region optional for non-service runner.
[lcwik] [BEAM-4374] Fix missing deletion of metrics.
[samuelw] [BEAM-9651] Prevent StreamPool and stream initialization livelock
[boyuanz] [BEAM-9562, BEAM-6274] Fix-up timers to use Elements.Timer proto in data
[robertwb] Allow unset write threshold for state backed iterable coder.
[github] Revert "[BEAM-9651] Prevent StreamPool and stream initialization
[github] [BEAM-9727] Automatically set required experiment flags for dataflow
[github] Update environments.py to add a method to specify container image
------------------------------------------
[...truncated 283.38 KB...]
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 10, 2020 1:18:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 1:18:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 10, 2020 1:18:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 10, 2020 1:18:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 10, 2020 1:18:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 10, 2020 1:18:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 10, 2020 1:18:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 10, 2020 1:18:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-SFVORjwRMTp705Jb-qvk9A.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-pe9FDnlamqoaSOgW1j3lqQ.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-jN9vO5UdvmQT9LisJ7PgLA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-hbKwAZVALeDXdbR3Efs6JA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-tkPvolamHp4IyfUZidlanw.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-7kVCuCTvz-NZmDy6W0VQJA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-HcUPQmr29zR-rlgIDaK9fA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-apayJBSFo_Oz0RUoGC78HA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-KToI0C8PbWbtM-o4MSZidw.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-hQmjGH9aHbmfsCUOE1JwcA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-WS87ZMep360jdo2ol3_UrA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-OER6nDtia_Lun4pq_ksLuw.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-AXUVNqiX_WNXYuUEJi68eA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-2Wadx17TgfZlLQzPKKnJIg.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-8eHtyk80h3w1-Qz8Tmy30g.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-rtt4NnepG65Aqo0wWgWG6w.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-VacybgQb_viBdcvrhQLyGw.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-l9JFaGxjMXmLvEWVzQBgtg.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-o_u-Iaa-5v8DnpH-R-ek2g.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-9SsKV1Jy65bmYSGRb9_TlA.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-7fE7Bp8PfCZFOEhSogvV9A.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-HhZGl4G0CHg46U-98lMXMw.jar
Apr 10, 2020 1:18:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-3GHSTuy5XbyIvIXkr64Ttw.jar
Apr 10, 2020 1:18:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-fzMvxXi6FEerJQ1hdqy1dQ.jar
Apr 10, 2020 1:18:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-97JoroXdhyUMgzy_b9DtZg.jar
Apr 10, 2020 1:18:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-bsD3gNubEafbX-Q0ctHACA.jar
Apr 10, 2020 1:18:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-evogubWNRvA4n2B_UujY9g.jar
Apr 10, 2020 1:18:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-bsD3gNubEafbX-Q0ctHACA.jar
Apr 10, 2020 1:18:27 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-bsD3gNubEafbX-Q0ctHACA.jar&uploadType=resumable&upload_id=AEnB2Uqv6rjA-sMzN4SQVx2V2FL1s83vnutnJKvls1W7cVlGrTSml6ReTQgT_MFVhxDEEoy4Zj5haqUh8MYqvMBsmVZM-Sed8u1pTYwrmb3jSJpUQMDCjoE.
Apr 10, 2020 1:18:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:309)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:215)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:272)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 10, 2020 1:18:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-bsD3gNubEafbX-Q0ctHACA.jar
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 10 seconds
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 10, 2020 1:18:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 10, 2020 1:18:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash AoDr6BxTwngVqnPsHTG1dw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-AoDr6BxTwngVqnPsHTG1dw.pb
Apr 10, 2020 1:18:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 10, 2020 1:18:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-09_18_18_36-18137486545759790478?project=apache-beam-testing
Apr 10, 2020 1:18:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-09_18_18_36-18137486545759790478
Apr 10, 2020 1:18:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-09_18_18_36-18137486545759790478
Apr 10, 2020 1:18:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T01:18:36.621Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 10, 2020 1:18:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:39.985Z: Checking permissions granted to controller Service Account.
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:44.986Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.541Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.569Z: Expanding GroupByKey operations into optimizable parts.
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.593Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.668Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.693Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.719Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:45.748Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:46.040Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 1:18:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:18:46.112Z: Starting 5 workers in us-central1-f...
Apr 10, 2020 1:19:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:19:11.293Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 10, 2020 1:19:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:19:11.317Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 10, 2020 1:19:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-10T01:19:13.557Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 10, 2020 1:19:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:19:16.801Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 10, 2020 1:19:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:19:32.349Z: Workers have started successfully.
Apr 10, 2020 1:19:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:19:32.389Z: Workers have started successfully.
Apr 10, 2020 1:20:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:20:08.964Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 10, 2020 1:20:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:20:09.152Z: Cleaning up.
Apr 10, 2020 1:20:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:20:09.236Z: Stopping worker pool...
Apr 10, 2020 1:22:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:21:59.894Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 10, 2020 1:22:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-10T01:21:59.937Z: Worker pool stopped.
Apr 10, 2020 1:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-09_18_18_36-18137486545759790478 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7b8b014c-f640-40f1-a21d-229fda0e5c3a and timestamp: 2020-04-10T01:22:05.455000000Z:
Metric: Value:
read_time 18.885
fields_read 4375276.0
Gradle Test Executor 25 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.002 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.005 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 49.876 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 20s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/7nkvis5ehtuoe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #364
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/364/display/redirect?page=changes>
Changes:
[github] [BEAM-8280] Document Python 3 annotations support (#11232)
[github] [BEAM-9731] Include more detail in passert.Equals errors. (#11359)
[github] [BEAM-9085] Fix performance regression in SyntheticSource on Python 3
------------------------------------------
[...truncated 285.59 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 09, 2020 7:08:59 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 09, 2020 7:08:59 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 09, 2020 7:08:59 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 09, 2020 7:08:59 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 09, 2020 7:09:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 09, 2020 7:09:00 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 09, 2020 7:09:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 09, 2020 7:09:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT--i1RMY_obbo4nnc1IdBDLQ.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-UTXuUt5hNTR-3uCAjbPfXA.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-Sd93lnoK6Q2-Ln-FvRdQWw.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-7H_s-8On23W6-ElaOFjtrg.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-W9deocpB361tke3dMBGSJw.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-xFoJiFeOEnI35V9NzOGITw.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-StTLz3vRihXnXsEkCJeUFQ.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-fFq1pbs7bS_1D73byLOS8A.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-RYGmjpLn-pp5evgNubMmwQ.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-qhIW0hy2Wk23ZHSY-6YL4Q.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-RIl9qNl2RLGTMcHqSdx5bw.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-WaPtVsmAUyClE7zauGCkbA.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-Fgen--kl3YcKOAWQczeYQg.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-ecDLkXqxliA9W2dKg3XPLw.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-PCIVgfPbIupKomwzwQxDzg.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-Gpp8nuOFhgdr1X6ctJlLtQ.jar
Apr 09, 2020 7:09:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-jkrzn6FDCFQmUay3CIMrow.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-PcO6znB8Tm-tx1ZtmKJdsA.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-Lt1ZRPokoqE0J-BYKypptA.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-1YROpaZuwGPCeGpf1PmWJg.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-qhIW0hy2Wk23ZHSY-6YL4Q.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-DoKe3akiA_qElC32y90XLA.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-AnAa8tz56-_AjQL3mO51pQ.jar
Apr 09, 2020 7:09:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-JtZwIRia_CPkoEvJ86rskg.jar
Apr 09, 2020 7:09:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tdh6hajtd8dt_RbcfdqikA.jar
Apr 09, 2020 7:09:07 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-qhIW0hy2Wk23ZHSY-6YL4Q.jar&uploadType=resumable&upload_id=AEnB2UoEl6pIp519ntdTcVR2Q3VwLWOThTEhfRs0bPoFqaMCrVs2o3FSpsYEuCE_ws72I-F45kUdSEUgTTGKeWdFSTTe3H4LfTD4NnepHFv_ncOvQ1PqMdc.
Apr 09, 2020 7:09:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:309)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:215)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:272)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 09, 2020 7:09:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-OYFkB9Ziag2PiSniGr-_Dw.jar
Apr 09, 2020 7:09:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-uJ462PC8F_-dzqo98N2XRA.jar
Apr 09, 2020 7:09:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-PFRFXbMeyPRjELYZNNXEdA.jar
Apr 09, 2020 7:09:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-qhIW0hy2Wk23ZHSY-6YL4Q.jar
Apr 09, 2020 7:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 28 files newly uploaded in 10 seconds
Apr 09, 2020 7:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 09, 2020 7:09:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 09, 2020 7:09:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 09, 2020 7:09:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 09, 2020 7:09:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 09, 2020 7:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12101 bytes, hash 5YyDnKj-rxmO9-NEEW1haw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5YyDnKj-rxmO9-NEEW1haw.pb
Apr 09, 2020 7:09:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 09, 2020 7:09:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-09_12_09_15-8224463330328644237?project=apache-beam-testing
Apr 09, 2020 7:09:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-09_12_09_15-8224463330328644237
Apr 09, 2020 7:09:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-09_12_09_15-8224463330328644237
Apr 09, 2020 7:09:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-09T19:09:15.633Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 09, 2020 7:09:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:18.756Z: Checking permissions granted to controller Service Account.
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:22.718Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.312Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.343Z: Expanding GroupByKey operations into optimizable parts.
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.366Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.434Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.462Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.483Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 09, 2020 7:09:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.507Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 09, 2020 7:09:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.865Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 09, 2020 7:09:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:23.923Z: Starting 5 workers in us-central1-c...
Apr 09, 2020 7:09:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-09T19:09:43.475Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 09, 2020 7:09:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:09:45.694Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 09, 2020 7:10:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:10:06.344Z: Workers have started successfully.
Apr 09, 2020 7:10:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:10:06.381Z: Workers have started successfully.
Apr 09, 2020 7:10:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:10:47.375Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 09, 2020 7:10:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:10:47.588Z: Cleaning up.
Apr 09, 2020 7:10:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:10:47.693Z: Stopping worker pool...
Apr 09, 2020 7:11:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:11:57.029Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 09, 2020 7:11:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T19:11:57.063Z: Worker pool stopped.
Apr 09, 2020 7:12:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-09_12_09_15-8224463330328644237 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 273547e4-5b59-4033-af51-7ffb55ac0c8a and timestamp: 2020-04-09T19:12:02.610000000Z:
Metric: Value:
read_time 19.059
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
fields_read 4375276.0
3 tests completed, 2 failed
Finished generating test XML results (0.074 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.131 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 31.008 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 34s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/aoul4iug7bjwy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #363
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/363/display/redirect?page=changes>
Changes:
[kamil.wasilewski] [BEAM-9721] Add --region to Dataflow-based load tests
[kamil.wasilewski] [BEAM-9721] LoadTestConfig: handle --region parameter and put default
[github] [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. (#11193)
[github] [BEAM-9721]Conditionally add Dataflow region to Dataflow-based
[michael.jacoby] [BEAM-9647] fixes MQTT clientId to long
------------------------------------------
[...truncated 289.49 KB...]
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 09, 2020 12:54:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 09, 2020 12:54:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 09, 2020 12:54:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-XIeiHm0rUR20n5BXlYvJKw.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-NTyhDexq4Z2DCOB9t74cJQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-MnaUDdsqHu8pUydO4gbxBA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-kQ19Yv2bgVR_GANSrka_yQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-af-de2P0aCaExuR_dDBDmQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-rGmJb66U3_uHOi1vRIMsLg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-aIng7WmaFecpZRpRzu3mQg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-A51wGEz5fS8yINBrhGcHPQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-RBXH20zFYPLoKEMEOErNWg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-p4M2xRMOvL_Z-7JlbSMb_A.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-NbjOzgkWDF0RpX8oCp2OTQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-1pGulHQBciW_uRrmJR3omA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-OPpzuiRZlWmo_GlX1wc9HA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-ZJCHwXMuH6ddrgg9C5EpEw.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-4dkRP7mi9SSsoW0WDloDrg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-xuUAJ93cHlQickrWUq59Pg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-uzkkynJjFq66DahCc_SWLA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-14J101QhdqM8rCSLTp1bKw.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-UyL4GGRUM2ttMwrU1oxE8A.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-_YvAri0ganBW9X43vWtkdg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-feLVpH-HfP4tw9qqk3SsgQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.mongo/2.2.0/781d14f4e3d9eeb0b4c3e00a4ec165a04b3dc5c4/de.flapdoodle.embed.mongo-2.2.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.mongo-2.2.0-3bAjs_S7MZ7-neWHp_CgmQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-QYL9TF4an6h3zpoD3sARsg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/commons-compiler/3.0.11/f2a6ec7fbc929c9fc87ff8bb486c0574951c5b09/commons-compiler-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-compiler-3.0.11-bepx6Dd9e13zaxGldAayYQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.janino/janino/3.0.11/e699e368095379ba0402ea1780a87fcaea16e68f/janino-3.0.11.jar to gs://temp-storage-for-perf-tests/loadtests/staging/janino-3.0.11-pubTQ_GoxcNrGgSD5_X7qw.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-UuoGpJtBhztG1_qTtL3Zeg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-aXA8HiUcHqudoTdwZ3cjww.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/de.flapdoodle.embed/de.flapdoodle.embed.process/2.1.2/986b38302fa10018d5baccee7f655c31ee9afe5b/de.flapdoodle.embed.process-2.1.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/de.flapdoodle.embed.process-2.1.2-YrD3oe9Ff_itNus4wP3waQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-NxT0kkQ4Lwcx9k7udVhDwA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.7/30b13b7efc55b7feea667691509cf59902375001/antlr4-runtime-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-runtime-4.7-t59VAkIGs5viU54ez94MCg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-Bs6hUAlYzO7IFhB5N5uJ0g.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.glassfish/javax.json/1.0.4/3178f73569fd7a1e5ffc464e680f7a8cc784b85a/javax.json-1.0.4.jar to gs://temp-storage-for-perf-tests/loadtests/staging/javax.json-1.0.4-Vphw-XXe7rZpH8ubwCqVVQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/ST4/4.0.8/a1c55e974f8a94d78e2348fa6ff63f4fa1fae64/ST4-4.0.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/ST4-4.0.8-ozr8Njx2YI3X-Kcj8Tfb8Q.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.abego.treelayout/org.abego.treelayout.core/1.0.3/457216e8e6578099ae63667bb1e4439235892028/org.abego.treelayout.core-1.0.3.jar to gs://temp-storage-for-perf-tests/loadtests/staging/org.abego.treelayout.core-1.0.3-nIzvq2NgpnJWU3DVMR8PPA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr-runtime/3.5.2/cd9cd41361c155f3af0f653009dcecb08d8b4afd/antlr-runtime-3.5.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr-runtime-3.5.2-H7uuLLclMCB8ILeXvavQKQ.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-NxT0kkQ4Lwcx9k7udVhDwA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.alibaba/fastjson/1.2.49/b8f5baf6d1b47ce3f21c1bd77242cc7c95e22fee/fastjson-1.2.49.jar to gs://temp-storage-for-perf-tests/loadtests/staging/fastjson-1.2.49-z0P0MjduZT1wQpv9Ia-IMA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4/4.7/cd6df46532bccabd8127c18c9ca5ef481962e931/antlr4-4.7.jar to gs://temp-storage-for-perf-tests/loadtests/staging/antlr4-4.7-COwoTCR4e1f3u2wy6oa_VA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.mongodb/mongo-java-driver/3.9.1/d313237180bf9f2f82e12f503d9617e6b070f792/mongo-java-driver-3.9.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/mongo-java-driver-3.9.1-SIrJ2zP3qz8Zl5iCpu_6gg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-zJcsydgYrLs2z9oo4BYomg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna/4.0.0/9b3a11c613ec3fd3440af4103b12c3de82d38b6e/jna-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-4.0.0-oeIOSKNnBjAj25E3zrfGPA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-calcite-1_20_0/0.1/6d16a59dc771784789116607a04acd9a0ef91d16/beam-vendor-calcite-1_20_0-0.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-calcite-1_20_0-0.1-bAqPbFlwv7RdbvHyrAw_yg.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.ibm.icu/icu4j/58.2/db9fd4b4c189cf1518db14c67d14a2cfcfbe59f6/icu4j-58.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/icu4j-58.2-YF2KAnaigP9jMsO9JgcRgA.jar
Apr 09, 2020 12:54:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/net.java.dev.jna/jna-platform/4.0.0/deb6bf66918989b50209b8c9aaf3b2561af7f011/jna-platform-4.0.0.jar to gs://temp-storage-for-perf-tests/loadtests/staging/jna-platform-4.0.0-6WVyLGquTreCjQNLObQw0g.jar
Apr 09, 2020 12:54:25 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-NxT0kkQ4Lwcx9k7udVhDwA.jar&uploadType=resumable&upload_id=AEnB2Uokp0aoPBUtiT1Pg2xeS-wZhKs3owL6R0pGAD8fqqB3mXnJhCNxgxBK-uJYGrkIjyminn9HAaQju8i-pF1qVAKbpvyPxPkmKjEd0tgbAQB9DJRA4Rs.
Apr 09, 2020 12:54:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar>
java.io.IOException: Upload failed
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:309)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:215)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:554)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:272)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Apr 09, 2020 12:54:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-NxT0kkQ4Lwcx9k7udVhDwA.jar
Apr 09, 2020 12:54:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 162 files cached, 44 files newly uploaded in 8 seconds
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash njApK1lTy5CFVXEKUG8cMw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-njApK1lTy5CFVXEKUG8cMw.pb
Apr 09, 2020 12:54:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 09, 2020 12:54:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-09_05_54_32-4962255897533356622?project=apache-beam-testing
Apr 09, 2020 12:54:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-09_05_54_32-4962255897533356622
Apr 09, 2020 12:54:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-09_05_54_32-4962255897533356622
Apr 09, 2020 12:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-09T12:54:33.023Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 09, 2020 12:54:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:36.612Z: Checking permissions granted to controller Service Account.
Apr 09, 2020 12:54:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:40.864Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 09, 2020 12:54:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.525Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 09, 2020 12:54:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.560Z: Expanding GroupByKey operations into optimizable parts.
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.589Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.761Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.797Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.827Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:41.863Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:42.265Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 09, 2020 12:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:54:42.346Z: Starting 5 workers in us-central1-f...
Apr 09, 2020 12:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-09T12:54:48.898Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 09, 2020 12:55:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:55:10.446Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 09, 2020 12:55:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:55:10.480Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 09, 2020 12:55:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:55:15.927Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 09, 2020 12:55:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:55:31.640Z: Workers have started successfully.
Apr 09, 2020 12:55:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:55:31.681Z: Workers have started successfully.
Apr 09, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:56:06.036Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 09, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:56:06.220Z: Cleaning up.
Apr 09, 2020 12:56:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:56:06.312Z: Stopping worker pool...
Apr 09, 2020 12:57:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:57:34.514Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 09, 2020 12:57:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-09T12:57:34.563Z: Worker pool stopped.
Apr 09, 2020 12:57:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-09_05_54_32-4962255897533356622 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2bb185ec-0027-4fa5-b66b-ef3cecdb8ba2 and timestamp: 2020-04-09T12:57:40.516000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.555
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.041 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 27.476 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 24s
101 actionable tasks: 83 executed, 18 from cache
Publishing build scan...
https://gradle.com/s/o7vkia5id2kde
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #362
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/362/display/redirect?page=changes>
Changes:
[samuelw] Ensure that empty messages are not flushed to handler.
[chamikara] Adds nose back under packages needed for testing.
[robertwb] [BEAM-9618] Mark push registration as deprecated.
------------------------------------------
[...truncated 254.82 KB...]
> Task :runners:direct-java:shadowJar
Build cache key for task ':runners:direct-java:shadowJar' is 990f74283d7ac8e4385c877bb0de37d0
Caching disabled for task ':runners:direct-java:shadowJar': Caching has not been enabled for the task
Task ':runners:direct-java:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:direct-java:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 6 (includes project)
Total Time: 0.566s [566ms]
Average Time/Jar: 0.0943333333333s [94.3333333333ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.776 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 46a3a5f73822be4c2010aad21e5a4368
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@3f4ff3d2: {executionTime=14652, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=cneno5mkhrcetbqz7aocj6opqy, creationTime=1586389953276, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.287 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.068 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is d7980df1a0db496b30c13b8c29aa92a5
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@2ce32988: {executionTime=14933, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=cneno5mkhrcetbqz7aocj6opqy, creationTime=1586389953556, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.415 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 1c7013d92543206b47133fe98665caef
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@296dea6e: {executionTime=34612, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=bwselxpifbfafg3yuwswfgezzu, creationTime=1586390716627, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Phrase/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.102 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is a006f2176d34f3c2732f987e8b18bad2
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.04 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.154 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 3830caf172f5e9a184a111e8fb08db34
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 2.981s [2981ms]
Average Time/Jar: 0.1863125s [186.3125ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3.884 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is 4fd4f96be32f5be7f38cd9fb4687f32e
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@1d50fce0: {executionTime=21924, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=ovwnswdok5hmzeua3lt5me4vea, creationTime=1586393636703, identity=:sdks:java:extensions:sql:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:compileTestJava' from cache.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.525 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.095 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is fdda7bbec58f75e568aed97e10653e0a
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@ebfbe6f: {executionTime=3037, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=ovwnswdok5hmzeua3lt5me4vea, creationTime=1586393640071, identity=:sdks:java:extensions:sql:perf-tests:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' from cache.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.094 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses UP-TO-DATE
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is e6bcdd03641b042649c1b466faaf4536
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.014 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4.119 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/tlds4dq4ouzzc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #361
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/361/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9322] [BEAM-1833] Better naming for composite transform output
[kcweaver] [BEAM-9716] Alias zone to worker_zone and warn user.
[crites] Updates documentation for WINDOWED_VALUE coder.
[github] Update session.go
[github] Update stage.go
[github] Update server_test.go
[github] Update materialize.go
[github] Update materialize_test.go
[github] Update stage_test.go
[github] Update artifact.go
[github] Update provision.go
[github] Update retrieval.go
[github] Update staging.go
[github] Update translate.go
[github] Update datamgr.go
[github] Update datamgr_test.go
[github] Update logging.go
[github] Update logging_test.go
[github] Update monitoring.go
[github] Update session.go
[github] Update statemgr.go
[github] Update statemgr_test.go
[github] Update replace.go
[github] Update replace_test.go
[github] Update provision.go
[github] Update execute.go
[github] Update job.go
[github] Update translate.go
[github] Update translate.go
[github] Update job.go
[github] Update materialize.go
[kcweaver] [BEAM-9714] [Go SDK] Require --region flag in Dataflow runner.
[github] Update translate.go
[github] Update session.go
[github] Update materialize_test.go
[mxm] [BEAM-9596] Ensure metrics are available in PipelineResult when the
[crites] Uses iterable coder for windows and copies all of timestamp encoding
[github] Update session.go (#11352)
[github] [BEAM-9618] Java SDK worker support for pulling bundle descriptors.
[github] [Beam-9063]update documentation (#10952)
------------------------------------------
[...truncated 256.36 KB...]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 1.189 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':',5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 46a3a5f73822be4c2010aad21e5a4368
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@5b416840: {executionTime=14652, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=cneno5mkhrcetbqz7aocj6opqy, creationTime=1586389953276, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.434 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.134 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is d7980df1a0db496b30c13b8c29aa92a5
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@3ee6671b: {executionTime=14933, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=cneno5mkhrcetbqz7aocj6opqy, creationTime=1586389953556, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':',5,main]) completed. Took 0.661 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':',5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':',5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':',5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 1c7013d92543206b47133fe98665caef
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@2ec7ebc0: {executionTime=34612, hostName=apache-beam-jenkins-11, operatingSystem=Linux, buildInvocationId=bwselxpifbfafg3yuwswfgezzu, creationTime=1586390716627, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Phrase/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 0.268 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':',5,main]) completed. Took 0.205 secs.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is a006f2176d34f3c2732f987e8b18bad2
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.043 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is c96feaf359041eea9a5251856fa59564
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.979s [3979ms]
Average Time/Jar: 0.2486875s [248.6875ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 5.237 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is 4fd4f96be32f5be7f38cd9fb4687f32e
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 3.625 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 22.123 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.139 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is fdda7bbec58f75e568aed97e10653e0a
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.137 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3.085 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is ad56185bfd1ad3003351202db0cc952a
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 4.953 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 37s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/wzo5inxdbcplo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #360
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/360/display/redirect?page=changes>
Changes:
[mxm] [BEAM-9580] Allow Flink 1.10 processing timers to finish on pipeline
[github] [BEAM-9147] Add a VideoIntelligence transform to Java SDK (#11261)
[mxm] Revert "[BEAM-9580] Downgrade Flink version to 1.9 for Nexmark and
[mxm] [BEAM-9557] Fix strings used to verify test output
------------------------------------------
[...truncated 254.15 KB...]
> Task :runners:direct-java:shadowJar
Build cache key for task ':runners:direct-java:shadowJar' is d9cc5c3e6c09d19a247037a014013b25
Caching disabled for task ':runners:direct-java:shadowJar': Caching has not been enabled for the task
Task ':runners:direct-java:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:direct-java:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 6 (includes project)
Total Time: 0.569s [569ms]
Average Time/Jar: 0.0948333333333s [94.8333333333ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.774 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 9a1806ecf49b2b84bef0004bdf4b5843
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@737dea1c: {executionTime=14616, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=ecnfgbcjuzeqfdnvwrvfjqkanq, creationTime=1586297162501, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.242 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.058 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is a88909c106796bcc9b2dc4541c88c50c
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@6edb0f34: {executionTime=16034, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=ecnfgbcjuzeqfdnvwrvfjqkanq, creationTime=1586297163918, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.437 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 90b420f0cc0022b29ce4149895b9fb6f
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@17bf36db: {executionTime=49493, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=wk3djcvsanhddhigbqqfuvlvom, creationTime=1586298657062, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.161 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.057 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.153 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 4ee82da6d0cf4df517512b62ddcd9bbc
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.068s [3068ms]
Average Time/Jar: 0.19175s [191.75ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Daemon worker,5,main]) completed. Took 3.973 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is d58ee416fc5b5951f58a3ca83308e9ee
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@233bb33d: {executionTime=14060, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=psfh3ty5hjhzjfhlc57r2oo2ee, creationTime=1586307657422, identity=:sdks:java:extensions:sql:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:compileTestJava' from cache.
:sdks:java:extensions:sql:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 0.588 secs.
:sdks:java:extensions:sql:testClasses (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Daemon worker,5,main]) completed. Took 0.1 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 5fa03398ea5e5ccc2606bcaa30408c30
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@59a386a2: {executionTime=2397, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=psfh3ty5hjhzjfhlc57r2oo2ee, creationTime=1586307660070, identity=:sdks:java:extensions:sql:perf-tests:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' from cache.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 0.112 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses UP-TO-DATE
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is c5ef0a729226a453d993c77b2065a354
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.016 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3.969 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/56ivse5spldt4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #359
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/359/display/redirect>
Changes:
------------------------------------------
[...truncated 254.35 KB...]
> Task :runners:direct-java:shadowJar
Build cache key for task ':runners:direct-java:shadowJar' is c5281b5395f61119ef77d71ba4e83667
Caching disabled for task ':runners:direct-java:shadowJar': Caching has not been enabled for the task
Task ':runners:direct-java:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:direct-java:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 6 (includes project)
Total Time: 0.57s [570ms]
Average Time/Jar: 0.095s [95.0ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.761 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 9a1806ecf49b2b84bef0004bdf4b5843
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@3266d7db: {executionTime=15220, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=ihn5ljaxkrab5as7jqtifatysa, creationTime=1586297392884, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.256 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.054 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is a88909c106796bcc9b2dc4541c88c50c
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@166ea520: {executionTime=11427, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=cztircktiffyzncktccqddaycm, creationTime=1586300285715, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.446 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 90b420f0cc0022b29ce4149895b9fb6f
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@6d3f3439: {executionTime=8595, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=zf3wgknmibd4hmektmfsflttzm, creationTime=1586302321679, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.167 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.042 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.133 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 30d4bb4c678235d92a263f6d9e46b8ff
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.04s [3040ms]
Average Time/Jar: 0.19s [190.0ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3.871 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is d58ee416fc5b5951f58a3ca83308e9ee
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@18119225: {executionTime=12408, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=xzh5b663wffkpfwuddysp74c7a, creationTime=1586328314488, identity=:sdks:java:extensions:sql:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:compileTestJava' from cache.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.525 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.096 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 5fa03398ea5e5ccc2606bcaa30408c30
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@6d32da56: {executionTime=2022, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=xzh5b663wffkpfwuddysp74c7a, creationTime=1586328316722, identity=:sdks:java:extensions:sql:perf-tests:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src}>
Unpacked trees for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' from cache.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.098 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses UP-TO-DATE
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is a03042eec74a87a3447b2899afaa0141
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.016 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3.764 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 47s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/yxghxbjidhdbc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #358
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/358/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-5422] Document DynamicDestinations.getTable uniqueness requirement
[github] [BEAM-9529] Remove datastore.v1, googledatastore (#11175)
------------------------------------------
[...truncated 254.61 KB...]
Total Jars: 6 (includes project)
Total Time: 0.59s [590ms]
Average Time/Jar: 0.09833333333330001s [98.3333333333ms]
*******************
:runners:direct-java:shadowJar (Thread[Daemon worker,5,main]) completed. Took 0.765 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon worker,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 9a1806ecf49b2b84bef0004bdf4b5843
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@259c52ff: {executionTime=15220, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=ihn5ljaxkrab5as7jqtifatysa, creationTime=1586297392884, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 0.28 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon worker,5,main]) completed. Took 0.064 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is a88909c106796bcc9b2dc4541c88c50c
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@15b7afd8: {executionTime=11427, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=cztircktiffyzncktccqddaycm, creationTime=1586300285715, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.436 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 90b420f0cc0022b29ce4149895b9fb6f
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@36cc6e39: {executionTime=8595, hostName=apache-beam-jenkins-6, operatingSystem=Linux, buildInvocationId=zf3wgknmibd4hmektmfsflttzm, creationTime=1586302321679, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 0.165 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon worker,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon worker,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon worker,5,main]) completed. Took 0.026 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.181 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 4626d7771e860b27a99ebc0ebd522727
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.078s [3078ms]
Average Time/Jar: 0.192375s [192.375ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3.933 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is d58ee416fc5b5951f58a3ca83308e9ee
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 1.908 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 12.531 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 0.094 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 5fa03398ea5e5ccc2606bcaa30408c30
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.087 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 2.025 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 11,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is a76783f32cfd81fa93e0b7034aa555d3
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.014 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3.619 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 0s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/lrqehfuyldhtc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #357
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/357/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-9691] Ensuring BQSource is avoided on FnApi
[pabloem] [BEAM-9715] Ensuring annotations_test passes in all
[github] Name the pipeline_v1 proto import
[github] Update materialize_test.go
[ankurgoenka] [BEAM-9707] Hardcode Unified harness image for fixing dataflow VR 2
[rohde.samuel] Fix flaky interactive_runner_test
[github] Merge pull request #11205 [BEAM-9578] Defer expensive artifact
[robertwb] Update go protos.
[github] Fix some Go SDK linter/vet warnings. (#11330)
[robertwb] [BEAM-9577] Plumb resources through Python job service and runner.
[robertwb] [BEAM-9618] Pull bundle descriptors for Go.
------------------------------------------
[...truncated 256.63 KB...]
Total Jars: 6 (includes project)
Total Time: 0.579s [579ms]
Average Time/Jar: 0.0965s [96.5ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.793 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 10,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 9a1806ecf49b2b84bef0004bdf4b5843
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@6304dc54: {executionTime=14616, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=ecnfgbcjuzeqfdnvwrvfjqkanq, creationTime=1586297162501, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.279 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.073 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is a88909c106796bcc9b2dc4541c88c50c
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@291a9aec: {executionTime=16034, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=ecnfgbcjuzeqfdnvwrvfjqkanq, creationTime=1586297163918, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.52 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 90b420f0cc0022b29ce4149895b9fb6f
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@1ede1c84: {executionTime=49493, hostName=apache-beam-jenkins-4, operatingSystem=Linux, buildInvocationId=wk3djcvsanhddhigbqqfuvlvom, creationTime=1586298657062, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.235 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 10,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 0.041 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.185 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 5d09dee5426cd0d31a31d0d5797d44c3
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.095s [3095ms]
Average Time/Jar: 0.1934375s [193.4375ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3.97 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is d58ee416fc5b5951f58a3ca83308e9ee
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 2.642 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 14.21 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.107 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 5fa03398ea5e5ccc2606bcaa30408c30
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.089 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 2.402 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is ba7af2ba05d9e39e5cf2e67efa8bc74e
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.015 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3.829 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 7s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/et3pm27nrxgkw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #356
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/356/display/redirect?page=changes>
Changes:
[github] Merge pull request #11226: [BEAM-9557] Fix timer window boundary
[github] Merge pull request #11244 from [BEAM-3097] _ReadFromBigQuery supports
------------------------------------------
[...truncated 255.98 KB...]
Total Jars: 6 (includes project)
Total Time: 0.934s [934ms]
Average Time/Jar: 0.15566666666669998s [155.6666666667ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 1.172 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 6,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 969b567238746d8bb2e69f171f6837fa
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@51a066f3: {executionTime=16787, hostName=apache-beam-jenkins-3, operatingSystem=Linux, buildInvocationId=dyjupjkq3vgetippdvndtchcki, creationTime=1586282758976, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_SQL/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.335 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.101 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 827855a302a4141c57edaa49ffaa99b5
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@6392d37c: {executionTime=10783, hostName=apache-beam-jenkins-3, operatingSystem=Linux, buildInvocationId=2mmgvqbwlbegtat5fomiefezwa, creationTime=1586284522667, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_XmlIOIT/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.222 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.035 secs.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is 6facf83edf4c31b366dd9313a762aaee
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@62c9467f: {executionTime=81740, hostName=apache-beam-jenkins-3, operatingSystem=Linux, buildInvocationId=bzh2gxb7vzddvlx47xoanyz6fu, creationTime=1586261427180, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Cron/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.714 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.157 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is af1d6f0a6bed9d54a80a4195fa5739e0
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.9s [3900ms]
Average Time/Jar: 0.24375s [243.75ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4.767 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is f67093d0f3117668cc16911cdb8f4fbc
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 3.101 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 18.419 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.108 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is e2ce2527e8c501ccba893b376f824ae1
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.119 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3.019 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is 38a57d62dd521c6ebdc90e773db5eadb
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4.449 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 21s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/iahaeue2btmxk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #355
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/355/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5980] Change load-tests build to include spark-sql for spark
[echauchot] [BEAM-9436] avoid one flatmap step and a KV creation per element by
[ecapoccia] [BEAM-9434] Improve Spark runner reshuffle translation to maximize
------------------------------------------
[...truncated 256.57 KB...]
Total Jars: 6 (includes project)
Total Time: 0.623s [623ms]
Average Time/Jar: 0.1038333333333s [103.8333333333ms]
*******************
:runners:direct-java:shadowJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.844 secs.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is c39b006b816cdd6e4d4aa3c97c2cef61
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@2a27b5d: {executionTime=16371, hostName=apache-beam-jenkins-7, operatingSystem=Linux, buildInvocationId=txzzeumwzncxloxujbhouc57ym, creationTime=1586241656037, identity=:sdks:java:io:google-cloud-platform:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Streaming_Performance_Test_Java/src}
Unpacked trees for task ':sdks:java:io:google-cloud-platform:compileTestJava' from cache.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.255 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses UP-TO-DATE
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.059 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava FROM-CACHE
Build cache key for task ':sdks:java:extensions:sql:compileJava' is 6facf83edf4c31b366dd9313a762aaee
Task ':sdks:java:extensions:sql:compileJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@39ab45b1: {executionTime=10970, hostName=apache-beam-jenkins-7, operatingSystem=Linux, buildInvocationId=hxj4ej7nyzempo3nvdbv2nakxi, creationTime=1586244423959, identity=:sdks:java:extensions:sql:compileJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_Nexmark_Dataflow/src}
Unpacked trees for task ':sdks:java:extensions:sql:compileJava' from cache.
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.404 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 7767523345e94cace7536b6838096b2a
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Origin for org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@7961a5b9: {executionTime=8442, hostName=apache-beam-jenkins-7, operatingSystem=Linux, buildInvocationId=txzzeumwzncxloxujbhouc57ym, creationTime=1586241664698, identity=:runners:google-cloud-dataflow-java:compileTestJava, type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution, userName=jenkins, gradleVersion=5.2.1, rootPath=/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Streaming_Performance_Test_Java/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava' from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.208 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.028 secs.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.178 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is 1cb4470f0b295926a0b70c0035ec2668
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.409s [3409ms]
Average Time/Jar: 0.2130625s [213.0625ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Daemon worker,5,main]) completed. Took 4.343 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is 24cd970036fb80d521ccb41c528671db
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 2.047 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 15.014 secs.
:sdks:java:extensions:sql:testClasses (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Daemon worker,5,main]) completed. Took 0.106 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 3f8ea34c6fd347c4584646b2d3ef6551
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.084 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Daemon worker,5,main]) completed. Took 2.336 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Daemon worker,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Daemon worker,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is 6dbaa8098c5798f0a80a252ca76c2647
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.017 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3.958 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 9s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/vrnrhufy5stvu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #354
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/354/display/redirect?page=changes>
Changes:
[alex] Add Beam Schema Options to changelog
[alex] [BEAM-9704] Deprecate FieldType metadata
[eekkaaadrian] [BEAM-9705] Go sdk add value length validation checking on write to
------------------------------------------
[...truncated 262.08 KB...]
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.065 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :sdks:java:extensions:sql:compileJava
Created classpath snapshot for incremental compilation in 0.669 secs. 2533 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileJava'
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 17.919 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 5,5,main]) started.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 0.155 secs.
> Task :runners:google-cloud-dataflow-java:compileTestJava
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 7767523345e94cace7536b6838096b2a
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':runners:google-cloud-dataflow-java:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.156 secs. 302 duplicate classes found in classpath (see all with --debug).
Packing task ':runners:google-cloud-dataflow-java:worker:legacy-worker:compileJava'
:runners:google-cloud-dataflow-java:worker:legacy-worker:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 21.201 secs.
:runners:google-cloud-dataflow-java:worker:legacy-worker:classes (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:classes
Skipping task ':runners:google-cloud-dataflow-java:worker:legacy-worker:classes' as it has no actions.
:runners:google-cloud-dataflow-java:worker:legacy-worker:classes (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
Build cache key for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is e79a40b3ed2f4bbb0cf4d123dd6b831b
Caching disabled for task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
> Task :runners:google-cloud-dataflow-java:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.204 secs. 1576 duplicate classes found in classpath (see all with --debug).
Packing task ':runners:google-cloud-dataflow-java:compileTestJava'
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 11.529 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 0.027 secs.
> Task :runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.344s [3344ms]
Average Time/Jar: 0.209s [209.0ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 4.181 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is 24cd970036fb80d521ccb41c528671db
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.753 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 14.497 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.098 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 3f8ea34c6fd347c4584646b2d3ef6551
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.069 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 2.605 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is 277532412682904182ffad9d9e450884
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3.689 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 45s
101 actionable tasks: 85 executed, 16 from cache
Publishing build scan...
https://gradle.com/s/c37c6bagp2uag
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #353
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/353/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9199] Require Dataflow --region in Python SDK.
[kcweaver] Add --region to tests where needed.
[kcweaver] [BEAM-9199] Require --region option for Dataflow in Java SDK.
[kcweaver] Add --region to Java GCP tests.
[kcweaver] Fix DataflowRunnerTest.
[kcweaver] Fix more Java unit tests missing --region.
[kcweaver] Add --region to DF streaming example tests.
[kcweaver] Add unit tests for get_default_gcp_region
[kcweaver] Add --region to Dataflow runner webpage.
[kcweaver] lint
[kcweaver] Add --region to more Java tests and examples.
[kcweaver] Add --region to more Python tests and examples.
[kcweaver] format
[kcweaver] Remove unrecognized --region option from non-DF tests.
[robertwb] [BEAM-9618] Add protocol for requesting process bundle descriptors.
[robertwb] [BEAM-9618] Update Python to support process bundle descriptor fetching.
[robertwb] [BEAM-9618] Java FnApiClient support for process bundle descriptor
[robertwb] Typo fix.
[github] remove nose (#11307)
[lcwik] [BEAM-4374, BEAM-6189] Delete and remove deprecated Metrics proto
[github] [BEAM-9685] remove Go SDK container from release process (#11308)
------------------------------------------
[...truncated 262.19 KB...]
Task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar' is not up-to-date because:
No history is available.
Custom actions are attached to task ':runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.202s [3202ms]
Average Time/Jar: 0.200125s [200.125ms]
*******************
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 4.275 secs.
> Task :sdks:java:extensions:sql:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.797 secs. 2533 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileJava'
:sdks:java:extensions:sql:compileJava (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 17.914 secs.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:extensions:sql:classes
Skipping task ':sdks:java:extensions:sql:classes' as it has no actions.
:sdks:java:extensions:sql:classes (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:extensions:sql:jar
Build cache key for task ':sdks:java:extensions:sql:jar' is c91bcab78f4538c7d928bea06de223d1
Caching disabled for task ':sdks:java:extensions:sql:jar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:jar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:jar (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 0.13 secs.
> Task :sdks:java:io:google-cloud-platform:compileTestJava
Created classpath snapshot for incremental compilation in 0.52 secs. 1576 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:io:google-cloud-platform:compileTestJava'
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 18.242 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is 6d2409cf32cc2596d61a3b729a499658
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar': Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.051 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 83e90d529bd8e33fa7d7c074edb7f5f1
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':runners:google-cloud-dataflow-java:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.077 secs. 1576 duplicate classes found in classpath (see all with --debug).
Packing task ':runners:google-cloud-dataflow-java:compileTestJava'
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 8.382 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is 4401c3c34c7ff4f4005c066bfc92de78
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar': Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.026 secs.
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:compileTestJava' is b96f4804849225b7c7a77b50187dcae7
Task ':sdks:java:extensions:sql:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.69 secs. 4077 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:compileTestJava'
:sdks:java:extensions:sql:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 14.359 secs.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:testClasses
Skipping task ':sdks:java:extensions:sql:testClasses' as it has no actions.
:sdks:java:extensions:sql:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:testJar
Build cache key for task ':sdks:java:extensions:sql:testJar' is 04efa19276372f6f20f1589525194ae1
Caching disabled for task ':sdks:java:extensions:sql:testJar': Caching has not been enabled for the task
Task ':sdks:java:extensions:sql:testJar' is not up-to-date because:
No history is available.
:sdks:java:extensions:sql:testJar (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.093 secs.
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:compileTestJava
Build cache key for task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is 2dd27eead9c43334d84d3f2b8e4923b9
Task ':sdks:java:extensions:sql:perf-tests:compileTestJava' is not up-to-date because:
No history is available.
All input files are considered out-of-date for incremental task ':sdks:java:extensions:sql:perf-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with error-prone compiler
Created classpath snapshot for incremental compilation in 0.065 secs. 3866 duplicate classes found in classpath (see all with --debug).
Packing task ':sdks:java:extensions:sql:perf-tests:compileTestJava'
:sdks:java:extensions:sql:perf-tests:compileTestJava (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 2.462 secs.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) started.
> Task :sdks:java:extensions:sql:perf-tests:testClasses
Skipping task ':sdks:java:extensions:sql:perf-tests:testClasses' as it has no actions.
:sdks:java:extensions:sql:perf-tests:testClasses (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest
Build cache key for task ':sdks:java:extensions:sql:perf-tests:integrationTest' is 66c451c0313659602766f1cc904b1c95
Task ':sdks:java:extensions:sql:perf-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Custom actions are attached to task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=sql_bqio_read_java_batch","--runner=DataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--workerHarnessContainerImage=","--dataflowWorkerJar=${dataflowWorkerJar}"] -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/5.2.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown FAILED
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.<init>(BigQueryIOPushDownIT.java:92)
Caused by:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by:
java.lang.IllegalArgumentException: Missing required values: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:246)
... 9 more
3 tests completed, 3 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3.56 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 10s
101 actionable tasks: 82 executed, 19 from cache
Publishing build scan...
https://gradle.com/s/3h25buwixguui
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #352
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/352/display/redirect?page=changes>
Changes:
[valentyn] Fixes platform-dependent assumptions in subprocess_server_test.py.
[valentyn] Switches a test helper to a Py3-version thereof.
[github] [BEAM-8019] Python SDK support for cross-langauge pipelines in Dataflow.
------------------------------------------
[...truncated 282.47 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 06, 2020 7:03:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 7:03:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 7:03:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 7:03:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 7:03:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 06, 2020 7:03:03 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-2iSoR2NRqrJXF7zZjqrBJA.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-PT5YY5fV6hGR-ZuMYtzKmQ.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-T1-FBv7tJPtO9kpGujsKIg.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-x94Sx2ooUT1D2yNaqvAe5A.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-PujMZpflweYG5aAPUcnrBg.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-bdHIOV267klMKodO-yuZGQ.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-AIT1jr6aot0Pdf26kRQ1ZQ.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-fyqiizgn9IscVtIkM5IuBg.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-rpY4w8SXvWpkS5CHFakAwQ.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-7YTz0OHA-uy35bulvsXYhg.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-HWFuXQUw2dPGdmqTY4qb4w.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-EqsQjj98nLKghFJ2hYPWoA.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-twQtyTyyHQsMhUwxYCXRgg.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-VwCSObMSQ3MCmT8XltrgyA.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-K8ijmKTdzsMyTsRAXq7mqA.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-0l7HhgUvLfpmIHV1Hr8cSA.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-hiLsUp8Ft8jbcqjlu6F60g.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-pT1omjafsPDi2382797d7A.jar
Apr 06, 2020 7:03:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-tn3FeUp12HMxRBdShTLo3A.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT--1zPxjBprrZvkVkJkXIunA.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-sjcU87JNQWExYtvgBa90RA.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-TLKrTNdS4FM7ishUNRDzVg.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-4DEU_gy5QY3FSp8WuIfskA.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-3DCltTLbYx2w_Yr6pfM3pg.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-_O6K9kd64zo9n-F3Hdk-Ig.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-vtIQnPmJtwUWxwGy_ZrzBQ.jar
Apr 06, 2020 7:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-WesfJbUrBIGZA6IqMCvbhQ.jar
Apr 06, 2020 7:03:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 06, 2020 7:03:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash LrOjikjJwwIbMqcvQ5exEA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LrOjikjJwwIbMqcvQ5exEA.pb
Apr 06, 2020 7:03:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 06, 2020 7:03:08 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 06, 2020 7:03:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-06_12_03_08-8566460805218721051?project=apache-beam-testing
Apr 06, 2020 7:03:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-06_12_03_08-8566460805218721051
Apr 06, 2020 7:03:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-06_12_03_08-8566460805218721051
Apr 06, 2020 7:03:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T19:03:08.816Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 06, 2020 7:03:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:12.785Z: Checking permissions granted to controller Service Account.
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:17.248Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.077Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.108Z: Expanding GroupByKey operations into optimizable parts.
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.137Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.242Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.269Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.301Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 06, 2020 7:03:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.341Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 06, 2020 7:03:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.741Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 7:03:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:18.828Z: Starting 5 workers in us-central1-f...
Apr 06, 2020 7:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:45.689Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 7:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:45.725Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 06, 2020 7:03:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T19:03:46.898Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 06, 2020 7:03:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:03:51.149Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 7:04:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:04:05.816Z: Workers have started successfully.
Apr 06, 2020 7:04:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:04:05.858Z: Workers have started successfully.
Apr 06, 2020 7:04:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:04:40.933Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 7:04:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:04:41.132Z: Cleaning up.
Apr 06, 2020 7:04:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:04:41.221Z: Stopping worker pool...
Apr 06, 2020 7:06:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:06:38.379Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 06, 2020 7:06:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T19:06:38.429Z: Worker pool stopped.
Apr 06, 2020 7:06:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-06_12_03_08-8566460805218721051 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c3011f49-d07e-44b8-a9b6-8729d1d2a97d and timestamp: 2020-04-06T19:06:45.010000000Z:
Metric: Value:
read_time 16.088
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 50.757 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 12s
101 actionable tasks: 71 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/iy5u6lsisozlo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #351
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/351/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9509] Improve error message for bad job server URL.
[mxm] [BEAM-8201] Cleanup FnServices from DockerEnvironmentFactory and
[iemejia] [website] Update information about Beam's LTS policy
------------------------------------------
[...truncated 281.88 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 12:48:06 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 12:48:07 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 06, 2020 12:48:07 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 06, 2020 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 06, 2020 12:48:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 06, 2020 12:48:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-_icMxeXFWHdDV56knIuJag.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-svvqVup_60qHqrkzgmeQxw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-qxUXLS2n1S_jkw8Z0Auj4g.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-WKduj52WIJYoaLJRoDz4iw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-wA-O0z2lNG1g6_LTL0P24A.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-aQVbVVJib8vpTFCs8j4nnA.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-AeZnQzfl4Fa1ZZxAVElZcA.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-3RJhUMCAaToj_4_NTyJsWg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-ou8CAgDGVZAOJyaoaZUCJw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-VPAxHOGtuV7YzYyL4eIhDg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-ARtjiJ768MUEy4erDTunpg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-PzyVLXtey-LWyu_ybnYIbQ.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-wxtJsqpUum_VtFvCVKC4Pg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-cko8oGuqbcvWI4ZzDtRNqw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-da7DFH6OWbEg56gM_nalEw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-SBAN9keVftuLT4_ZzBXsKw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-tOn_BnZuAeEXor2AHjb9_A.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-jjSnb19krhKOtu4ZnUemvg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-f1E9U0iXbkBtZIZZcMc11A.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-f0S_EI1nJaYd0spNpq88Lg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-ci5unY6QhnRRbSH699ssbA.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-U15MceeYM6bkjDrn22cK-A.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-SgS2OIQGxpgw_FIdkCpT8g.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-jJ8tjUKM0f966ECJcQtWNQ.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-hnRcR-FtO-M6F6GjjdIyKg.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-Uv-q2qBfjrACE5RKOnpeAw.jar
Apr 06, 2020 12:48:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-gH18b81v3AuBzkS0B4MvvQ.jar
Apr 06, 2020 12:48:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 1 seconds
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash Gcki0GcIOo1jIgsI5dvirg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Gcki0GcIOo1jIgsI5dvirg.pb
Apr 06, 2020 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 06, 2020 12:48:11 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 06, 2020 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-06_05_48_11-5141620577697851819?project=apache-beam-testing
Apr 06, 2020 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-06_05_48_11-5141620577697851819
Apr 06, 2020 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-06_05_48_11-5141620577697851819
Apr 06, 2020 12:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T12:48:11.285Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 06, 2020 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:15.704Z: Checking permissions granted to controller Service Account.
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:19.819Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.534Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.581Z: Expanding GroupByKey operations into optimizable parts.
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.618Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.743Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.787Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.822Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:20.862Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:21.330Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:21.410Z: Starting 5 workers in us-central1-a...
Apr 06, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T12:48:44.498Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 06, 2020 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:48:49.408Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:49:08.175Z: Workers have started successfully.
Apr 06, 2020 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:49:08.222Z: Workers have started successfully.
Apr 06, 2020 12:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:49:40.615Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 12:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:49:40.835Z: Cleaning up.
Apr 06, 2020 12:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:49:40.915Z: Stopping worker pool...
Apr 06, 2020 12:50:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:50:56.518Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 06, 2020 12:50:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T12:50:56.575Z: Worker pool stopped.
Apr 06, 2020 12:51:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-06_05_48_11-5141620577697851819 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 78df1a63-a7ad-4264-b5c7-d1b1d0b86aa8 and timestamp: 2020-04-06T12:51:01.775000000Z:
Metric: Value:
read_time 15.571
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 3.605 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 11s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/gx3bal4gutuiu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #350
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/350/display/redirect>
Changes:
------------------------------------------
[...truncated 280.15 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 06, 2020 6:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 06, 2020 6:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 06, 2020 6:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 06, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 06, 2020 6:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-D2s4tQof-5KsiOL8p1okTQ.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-EnKrfMUE_oG06fyiHWqhEQ.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-Nvfl5ojAGXJfhIUjTehzHw.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-xLwwwWA3WAZy1CI8IhGjjA.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-TMqFQj92-uvezvHPgz-FAg.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-RuvZ0EOmMjOuNFEc_T2qEw.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-pHqU2WHPbex4psB_hH48FA.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-xfAwwhWYaQxRe3MSk8M69w.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Zuf6HwJmxRxHM7yCZ66law.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-RT0_LAMw1qWKBpSS3X-FEA.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-SiucHUbioYlCV5aVPSMM3w.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-3V2aepDAi9yMJx281Nk4Sw.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-QzdziTEqkgSNdwaRfCZevw.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-RzOv0hYuLUl0GlIE_dhLtg.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-z_zp9EMKgQmKriV1EmA8RQ.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-nPZhELK5egQ4sz4jLDWXxA.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-NObwozm24xUcXdgvnUpgtw.jar
Apr 06, 2020 6:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-_wVY6l6q0OTaQ1cm4u2EMA.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-i8dXgQOboVoMdKvE11StCQ.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-MN8Y5lRgHzDdd7VMsQhQDw.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-dDxqoJ_A-n8kTtlgLauSVw.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-zb005bSJx7KX0yFsZ23n1g.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-PyMUg6aGwmyyculdtFXj0w.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-U2dr6H5VWUUKeawz-B44uA.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT--3I28baEuZJZIaLjHN9rLQ.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-ItPCOzBV5ZbiSxtiDyQdCg.jar
Apr 06, 2020 6:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-L8TAiYHKj2x1Jnw1vtpxwQ.jar
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 06, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash g_qY5xq-QjX9wqXFI6NUIA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-g_qY5xq-QjX9wqXFI6NUIA.pb
Apr 06, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 06, 2020 6:45:21 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 06, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-05_23_45_21-11193007081729891687?project=apache-beam-testing
Apr 06, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-05_23_45_21-11193007081729891687
Apr 06, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-05_23_45_21-11193007081729891687
Apr 06, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T06:45:21.834Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 06, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:25.743Z: Checking permissions granted to controller Service Account.
Apr 06, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:30.218Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 06, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:30.871Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 06, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:30.905Z: Expanding GroupByKey operations into optimizable parts.
Apr 06, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:30.952Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.035Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.074Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.105Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.145Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.526Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:31.599Z: Starting 5 workers in us-central1-f...
Apr 06, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T06:45:52.937Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 06, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:55.810Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:45:55.842Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 06, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:01.203Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:14.830Z: Workers have started successfully.
Apr 06, 2020 6:46:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:14.864Z: Workers have started successfully.
Apr 06, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:52.893Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:53.172Z: Cleaning up.
Apr 06, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:46:53.269Z: Stopping worker pool...
Apr 06, 2020 6:48:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:48:43.951Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 06, 2020 6:48:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T06:48:43.994Z: Worker pool stopped.
Apr 06, 2020 6:48:51 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-05_23_45_21-11193007081729891687 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 436cbee7-ad0d-4312-9e20-70b845a7c630 and timestamp: 2020-04-06T06:48:51.467000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.42
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 43.731 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 33s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/dkxa74uaw4tq4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #349
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/349/display/redirect>
Changes:
------------------------------------------
[...truncated 278.15 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 06, 2020 12:45:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 12:45:23 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 12:45:23 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 06, 2020 12:45:23 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 06, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-m9lcurSuTs7LB0M4wyQ8Bg.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-7aTk6bS0pC55mw11FyM38g.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-GT3_4NfJwktQTu1ycSUoQA.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-NYajm2VZUBLXIGzDmHPv9w.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-rPlOtogZm7u_ziYlkECy_w.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-L5y1nnxsagA9KO4j6k7k9g.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-zb2p5ZxIRESYG0mWCAp82Q.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-U_KFiK9Tp5o0jbClrhXSvA.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-SGL-L4CVjZByr-xjwFrC_g.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-L2tEu050EMvkTM-WBRFZvQ.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-8tdzwz_XWX-IvU_a0Edldw.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-m9WbAI6A1fSNS47mR4zTQw.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-AMQlC6PhRz2dPTcHMA2Dag.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-_lJ_oeQIr1WyUNR4t6NCgQ.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-pqPWJjorw7EZQ7mXtpDRmQ.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-cSWpGaUXvrySt4g1D5rcww.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-1Db06UVeH-E5IJHxW6zyOw.jar
Apr 06, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-BJ6OM1-nvnx82UqS_chv5g.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-5Hnf3xELqnPuRVwcne1IqA.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-24Dji02YwTagwdMyVtGGHw.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-CwJ5p3-aLj3WI3hRgS8gnA.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-cKKwX4D5oKwnMUpt4Iom8w.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-O2esVtvjm2zZrJtRwuquVA.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-Vvqo3mpfkv6UirJ-3WoyyQ.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-wkQgTbUzKb551dxOfWAn7A.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-SKKlMkXzpaeqfAhOUz0i-A.jar
Apr 06, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-XJmdBJHqaQY6Kz4ssJTkPg.jar
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 06, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 06, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash aBcUZKBsMXCi_vo9fehvXw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aBcUZKBsMXCi_vo9fehvXw.pb
Apr 06, 2020 12:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 06, 2020 12:45:28 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 06, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-05_17_45_28-11499037293486801398?project=apache-beam-testing
Apr 06, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-05_17_45_28-11499037293486801398
Apr 06, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-05_17_45_28-11499037293486801398
Apr 06, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T00:45:28.967Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 06, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:32.628Z: Checking permissions granted to controller Service Account.
Apr 06, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:38.450Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.080Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.111Z: Expanding GroupByKey operations into optimizable parts.
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.144Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.232Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.252Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.287Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.323Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.645Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:45:39.721Z: Starting 5 workers in us-central1-a...
Apr 06, 2020 12:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-06T00:45:54.160Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 06, 2020 12:46:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:46:13.077Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 06, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:46:29.640Z: Workers have started successfully.
Apr 06, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:46:29.671Z: Workers have started successfully.
Apr 06, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:47:03.586Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 06, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:47:03.824Z: Cleaning up.
Apr 06, 2020 12:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:47:03.895Z: Stopping worker pool...
Apr 06, 2020 12:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:48:42.146Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 06, 2020 12:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-06T00:48:42.180Z: Worker pool stopped.
Apr 06, 2020 12:48:47 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-05_17_45_28-11499037293486801398 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 988fadbe-8fc3-4b3b-92be-c95bba0c7fd7 and timestamp: 2020-04-06T00:48:48.035000000Z:
Metric: Value:
read_time 16.929
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 32.303 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 30s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/tgoy6gyxrd3ds
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #348
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/348/display/redirect>
Changes:
------------------------------------------
[...truncated 281.46 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 05, 2020 7:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 05, 2020 7:50:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 05, 2020 7:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 05, 2020 7:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-_1qEJAy31qQe9dSK7y5HpQ.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-eyAZEgcMne2KBs-XJwFG2Q.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-Tven3sIz8HwnOWhjq4z5Yw.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-n1MCsvZpo1WkNGUqHF-K0w.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-6nPui1bdcmr6-_g9d4kUug.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-WOidSt3VV632S6npk3dlvg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-Hly9LxplxCXynKxKvvXl1Q.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-DQ0ow98J-ITLXPkKrDogGg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-R_eGCBt7yLjVIajZqql5GQ.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-wOueiAh2QX-6_cTrFwsKuw.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-6pFwImRHoSXS8dWcg83nGQ.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-Jp8B9j2p5XjBYCvhtKlqQQ.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-iYDq2H1jgZcoeiMam_03Wg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-RKmbFTj8IRxs4nwfPcj5Tg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-MbEtaTLQ-_cjpILpx9i7cg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-dZUbsch_BJDWA530orAsSg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-uJRUe5f1YywyMP4RtW61Zw.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-FK26NUChpWsLH6WHGf9UPA.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-Kr0xNSqxENSBvrX860agzA.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-4UZNJn1UFvFwIaIipesPmg.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-h3nMXl5PXMq99d9_sMAMHA.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-OS5CecfzGVESa1slGOqODA.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-yUxvP6xRF6z7_hFc63Ta3g.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-wmWyilH31ht7X-GYeB0OKQ.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-p9joPN-MglZ_WgYd1_fwPA.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-AzjOnnXYKo8EESlR2JfE5Q.jar
Apr 05, 2020 7:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-581XbtQy7y6fVpy0pv0iew.jar
Apr 05, 2020 7:50:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 05, 2020 7:50:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash MiiWsrQMp830_PuF8xxDNw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-MiiWsrQMp830_PuF8xxDNw.pb
Apr 05, 2020 7:50:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 05, 2020 7:50:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 05, 2020 7:50:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-05_12_50_07-6194898640339175681?project=apache-beam-testing
Apr 05, 2020 7:50:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-05_12_50_07-6194898640339175681
Apr 05, 2020 7:50:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-05_12_50_07-6194898640339175681
Apr 05, 2020 7:50:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T19:50:07.750Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 05, 2020 7:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:11.253Z: Checking permissions granted to controller Service Account.
Apr 05, 2020 7:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:15.856Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 05, 2020 7:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.485Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 05, 2020 7:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.528Z: Expanding GroupByKey operations into optimizable parts.
Apr 05, 2020 7:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.558Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 05, 2020 7:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.644Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 05, 2020 7:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.681Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 05, 2020 7:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.719Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 05, 2020 7:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:16.753Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 05, 2020 7:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:17.201Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 7:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:17.289Z: Starting 5 workers in us-central1-a...
Apr 05, 2020 7:50:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T19:50:32.818Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 05, 2020 7:50:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:50:42.795Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 05, 2020 7:51:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:51:02.578Z: Workers have started successfully.
Apr 05, 2020 7:51:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:51:02.607Z: Workers have started successfully.
Apr 05, 2020 7:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:51:40.772Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 7:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:51:40.989Z: Cleaning up.
Apr 05, 2020 7:51:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:51:41.089Z: Stopping worker pool...
Apr 05, 2020 7:53:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:53:33.984Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 05, 2020 7:53:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T19:53:34.041Z: Worker pool stopped.
Apr 05, 2020 7:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-05_12_50_07-6194898640339175681 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 69c3c218-413d-4a7b-aed1-b01ef7fc76a4 and timestamp: 2020-04-05T19:53:41.059000000Z:
Metric: Value:
read_time 18.953
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 47.083 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/7au3thrp2jheu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #347
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/347/display/redirect?page=changes>
Changes:
[robertwb] ResolveArtifact -> ResolveArtifacts
[robertwb] Regenerate protos.
[github] Merge pull request #11259: Use attachValues in SQL
------------------------------------------
[...truncated 280.20 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 05, 2020 6:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 6:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 6:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 6:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 05, 2020 6:45:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-LO--auhJkMqu-nZJ5Wx1Dg.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-KctR8NfF9PIUEX-qmMuAvA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-hMQnxj_P5X5K46plJwKZoA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-093s5tuA1JEjPn5U0JopgA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-AfSWtSqWJCoKIoMXo-iUMA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-ZuMDljgtSEWn_kw7gYY3nA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-1lCK4PgCHy9MZ5zacWzI3g.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-Oy9heGYbZbpfd0UKVyrYNQ.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-r7g8I1c3Pdz7rYCA3GzUEQ.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-a_GZE7ucCJ7IQhAksTM4qA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-fnX_Bwj3fdCM9TNvxCRZBw.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-MQ3Lqos6embzTeOdCcJ-wA.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-vDYiHTVP3-1QCoGQ__AnLQ.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-x1cIOHl5J4e9dRa3Axroxg.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-f-sATm6IleER_UleJ6LhOg.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-IWAQgE53Y5QfbMXbi-QI4A.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-XsRsvhQ-pdRUkeMV7sk8dw.jar
Apr 05, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-xhAG2aLgTBngYjhMSEVlJA.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-Qh_U7Fz-gou45Shj2o3Q0A.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-IK-aWv7eHuo1IdABWZaY7w.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-ZVp2StQfrAg_fM4uxVAyEw.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-_yM-GrDiYanOwkc_acBTuA.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-9tQsgnmhvuIMwk68u-fmIw.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-DcbvCQdTxqTWJfnVLPbrZA.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-aq4g-NFMwLVS4qFqyG_9xA.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-KlVSjL3FfG3U3Tige2gUyg.jar
Apr 05, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-mhQYzRggoXHxYJ_GaTUbng.jar
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 05, 2020 6:45:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 05, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12099 bytes, hash rwTKi8NXgUiksELBEtwbdQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rwTKi8NXgUiksELBEtwbdQ.pb
Apr 05, 2020 6:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 05, 2020 6:45:29 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 05, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-04_23_45_29-17815278321824073615?project=apache-beam-testing
Apr 05, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-04_23_45_29-17815278321824073615
Apr 05, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-04_23_45_29-17815278321824073615
Apr 05, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T06:45:29.960Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 05, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:33.539Z: Checking permissions granted to controller Service Account.
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:37.178Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:37.842Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:37.884Z: Expanding GroupByKey operations into optimizable parts.
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:37.925Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.019Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.053Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.088Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.121Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.588Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:45:38.661Z: Starting 5 workers in us-central1-f...
Apr 05, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T06:45:58.909Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 05, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:02.311Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 05, 2020 6:46:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:02.340Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 05, 2020 6:46:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:07.714Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 05, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:20.247Z: Workers have started successfully.
Apr 05, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:20.279Z: Workers have started successfully.
Apr 05, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:51.766Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:51.953Z: Cleaning up.
Apr 05, 2020 6:46:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:46:52.041Z: Stopping worker pool...
Apr 05, 2020 6:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:48:34.832Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 05, 2020 6:48:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T06:48:34.886Z: Worker pool stopped.
Apr 05, 2020 6:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-04_23_45_29-17815278321824073615 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ab48583a-d607-4b9c-a206-b9ea05d2807a and timestamp: 2020-04-05T06:48:40.334000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.709
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 24.437 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 23s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/zem5cr3bglx4u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #346
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/346/display/redirect>
Changes:
------------------------------------------
[...truncated 279.16 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 05, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 05, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 12:45:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 05, 2020 12:45:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-0aW5sl85YzKhu-pRcTQcng.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-2oQe1KeBQ0XuDfBiuncyZw.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-ZTC0l7W8-OhHLK8vOGWSyw.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-gAADTCe7iWZaMlMpyY91ZQ.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-J9G4_GbqqCkESv95cYWMDA.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-ZarpA9xB7h6nLLTJU7hClQ.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-Ke_h5z7fl5OieyAiehZ2fA.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-HwBx_XMIjPS5rZVG2aJ86A.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-hJ9CofVoqDACdJ_pTY2nyg.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-xISfbacP8BRcLI0Fxqquhw.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-bXdxZJAdNQQNyPLt46HY9w.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-G915SWj_B6tzbrN9VXOEyg.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-PNEKnG2TAW5Kw7WFjLBCoQ.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-SFCtYi7Zi7-qwfOO_brG5Q.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-0jShmx6shx_Fx1nQPhv1Wg.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-MFL46H8WuOMjWpyWkq4ssQ.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tw80-OqCYFhQcg5yHYpbaQ.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-d0NfNYjGjM5mmPEA89qUJg.jar
Apr 05, 2020 12:45:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-QVObola2Gdl67VwvmXhBEw.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-MarStNPBJ6LFEMCwTqW4_g.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-MNGVy1XKdbJbSkKrnpQkrQ.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-J4AAJc3lQao1ldLFJAWpQQ.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-XUlEjepVLwER_a69LRWrow.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-A-Y_J5hdPVzceEeVrLKhFw.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-1PVz0X8VmPuxGelcfGq1IQ.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-nIjNvqP4UM52oNUeSG5P-A.jar
Apr 05, 2020 12:45:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-KwEefswk0kvp3lh2Wba8Lg.jar
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 05, 2020 12:45:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 05, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12082 bytes, hash Cbk3SpvR0cINBJ_IVjSEYA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Cbk3SpvR0cINBJ_IVjSEYA.pb
Apr 05, 2020 12:45:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 05, 2020 12:45:18 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 05, 2020 12:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-04_17_45_18-16992071467925314905?project=apache-beam-testing
Apr 05, 2020 12:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-04_17_45_18-16992071467925314905
Apr 05, 2020 12:45:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-04_17_45_18-16992071467925314905
Apr 05, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T00:45:19.004Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 05, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:23.251Z: Checking permissions granted to controller Service Account.
Apr 05, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:27.851Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 05, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.488Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 05, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.527Z: Expanding GroupByKey operations into optimizable parts.
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.566Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.632Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.662Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.694Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:28.726Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:29.468Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:29.550Z: Starting 5 workers in us-central1-a...
Apr 05, 2020 12:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-05T00:45:36.213Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 05, 2020 12:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:45:55.820Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 05, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:46:15.096Z: Workers have started successfully.
Apr 05, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:46:15.134Z: Workers have started successfully.
Apr 05, 2020 12:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:46:56.225Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 05, 2020 12:46:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:46:56.439Z: Cleaning up.
Apr 05, 2020 12:46:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:46:56.528Z: Stopping worker pool...
Apr 05, 2020 12:48:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:48:33.829Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 05, 2020 12:48:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-05T00:48:33.875Z: Worker pool stopped.
Apr 05, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-04_17_45_18-16992071467925314905 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
Load test results for test (ID): 46142a9c-a030-4de2-855e-321c15039c5f and timestamp: 2020-04-05T00:48:40.531000000Z:
Metric: Value:
read_time 22.634
fields_read 4375276.0
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 35.179 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 22s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/4nutzsbydfcie
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #345
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/345/display/redirect>
Changes:
------------------------------------------
[...truncated 279.20 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 04, 2020 6:46:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 6:46:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 6:46:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 6:46:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 04, 2020 6:46:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-rNvwyczz4iofHZBaCXFNAA.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-nWvZPWnwYbqk6UwBrQma2A.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-pnYDmlBU43k_cUygze9AVQ.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-AqiyG7XFPdTTDjBcoCWJhA.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-BSg2Y2hOqDmOD4Y1N82Fug.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-oMaFBq3v-CBf09rbA4gFng.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-4GraWMAv330FCLSDPB7aNw.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-6uYoaIFV91pj4-WUmW1fWQ.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-LLirBfIM5Q7sRa32Qj1KBg.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-AUzrdM-VGydejUvUIc-1nA.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-Fk8Cnj5XZOQJIy85-ztP-A.jar
Apr 04, 2020 6:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-OpGm0aoz8Pd00wPmmjCNYA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-kNfY-hy4kdjpUOGZxrx8UQ.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-5c2aD4p22DJ4jbDysW9XVQ.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-Vve84XgPmi_mv4xo4auBMA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-SNHyIenq1feTKGSFSsikQA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-ed6yBmIlb2LTuQ2YlJtE2A.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-WIbEbUmhT3ukYA2LE2Y0Tw.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-CmnP3Cf95-J63XZxMlVNuQ.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-pq2hDl0rzWgOyTj3pkxibg.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-03fKJG3bCpilcaZZw4waUA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-TDppx7IA3L5FavioDmPHsg.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-reaWkQzthqe7jNaQJumfQw.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-fUj7DVvalBBVvZfTTdRUpA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-xfW-Q2RV9CJ42O1ZMQTHZw.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-CyS7S6oxASJE53Vb5bTREA.jar
Apr 04, 2020 6:46:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-1RIQ3Bt8okGZzQ2oq-rQbQ.jar
Apr 04, 2020 6:46:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 04, 2020 6:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash ysGn_B-MXHNfhtlZfmXIHw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ysGn_B-MXHNfhtlZfmXIHw.pb
Apr 04, 2020 6:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 04, 2020 6:46:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 04, 2020 6:46:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-04_11_46_31-11013360181615211945?project=apache-beam-testing
Apr 04, 2020 6:46:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-04_11_46_31-11013360181615211945
Apr 04, 2020 6:46:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-04_11_46_31-11013360181615211945
Apr 04, 2020 6:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T18:46:31.128Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 04, 2020 6:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:34.805Z: Checking permissions granted to controller Service Account.
Apr 04, 2020 6:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:39.593Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.190Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.235Z: Expanding GroupByKey operations into optimizable parts.
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.274Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.357Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.390Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.426Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.463Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.863Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 6:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:46:40.938Z: Starting 5 workers in us-central1-a...
Apr 04, 2020 6:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T18:46:50.064Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 04, 2020 6:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:47:09.519Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:47:24.612Z: Workers have started successfully.
Apr 04, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:47:24.645Z: Workers have started successfully.
Apr 04, 2020 6:48:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:48:04.018Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 6:48:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:48:04.255Z: Cleaning up.
Apr 04, 2020 6:48:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:48:04.336Z: Stopping worker pool...
Apr 04, 2020 6:49:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:49:35.969Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 04, 2020 6:49:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T18:49:36.006Z: Worker pool stopped.
Apr 04, 2020 6:49:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-04_11_46_31-11013360181615211945 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0be8f2fc-26ae-4375-b685-fbd41a45131d and timestamp: 2020-04-04T18:49:41.782000000Z:
Metric: Value:
read_time 21.367
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.019 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 23.471 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 23s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/nufotbbflrxw2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #344
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/344/display/redirect>
Changes:
------------------------------------------
[...truncated 280.12 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 04, 2020 12:50:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 12:50:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 12:50:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 12:50:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 12:50:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 04, 2020 12:50:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-OcExxPTXAfJDdQxs-1HTnw.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-mdfiB1wzGkxy8Il2DD7h0Q.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-MIdCiqiUsAjRoSHSw-BatQ.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-4UABzWIA9yVo5QY_BM0cKA.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-TvBnufh8Xd_axdgKKxXv3g.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-YmU9g6VAVpWu27AVnJlYGg.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-n27SkNsx9_pZjI5XCjoZ4w.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-ZjMLY5H64tXMlsVtDz4tCw.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-GwifgaMCM0dMLM-wwjwylg.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-V5Ouq2JA94jfCnb4kv4saw.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-N0lYJ9xilabz7y94TMHP7Q.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-kIpmM2Pe_2vT65Qk37R23Q.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-OvsysKijVcY4OMRre4lauQ.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-_hnkYhP3eL2r2YjeZSXdtw.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-h5Ph-nsWR8QR6ZXZ-u_cMg.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-iqGhXfE-vjhfxWDpzltTHQ.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-IJBZGOs0AI8QsB-wOcEvjQ.jar
Apr 04, 2020 12:50:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-L__1eyrDh4szDe7-POfKFw.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-UNINyWm6iXuUSuGj-CAJQA.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-yjALaEPri3IMFV800cp5-w.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-7eMVMNtxYVSzWuPegbTkJQ.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Cjuh85YLmGD_swrUy2UK3w.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-pVfKB8Oaw-sh4wQlsnPUwg.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-qthuQXTvKx4nTv9t2TSbEw.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-mkljBgD8jLTH3Ss-EBVYCA.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-E2FTTkJ0HUwxZsUGOEnevA.jar
Apr 04, 2020 12:50:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-HFA3j0-3a97Gg4S9M4PrYA.jar
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 04, 2020 12:50:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 04, 2020 12:50:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash cR81xltCYHCnS3B6D9JCmw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cR81xltCYHCnS3B6D9JCmw.pb
Apr 04, 2020 12:50:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 04, 2020 12:50:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 04, 2020 12:50:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-04_05_50_07-10351392385497247643?project=apache-beam-testing
Apr 04, 2020 12:50:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-04_05_50_07-10351392385497247643
Apr 04, 2020 12:50:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-04_05_50_07-10351392385497247643
Apr 04, 2020 12:50:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T12:50:07.194Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 04, 2020 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:10.807Z: Checking permissions granted to controller Service Account.
Apr 04, 2020 12:50:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:14.763Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.511Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.552Z: Expanding GroupByKey operations into optimizable parts.
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.581Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.668Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.697Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.730Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:15.761Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:16.142Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:16.214Z: Starting 5 workers in us-central1-a...
Apr 04, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:42.876Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 12:50:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:42.904Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 04, 2020 12:50:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T12:50:46.653Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 04, 2020 12:50:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:50:48.287Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:51:02.442Z: Workers have started successfully.
Apr 04, 2020 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:51:02.475Z: Workers have started successfully.
Apr 04, 2020 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:51:41.753Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:51:41.951Z: Cleaning up.
Apr 04, 2020 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:51:42.042Z: Stopping worker pool...
Apr 04, 2020 12:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:53:24.705Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 04, 2020 12:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T12:53:24.754Z: Worker pool stopped.
Apr 04, 2020 12:53:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-04_05_50_07-10351392385497247643 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 96e68fbc-6573-4180-9260-138fafce9e32 and timestamp: 2020-04-04T12:53:30.645000000Z:
Metric: Value:
read_time 20.341
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 36.781 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 41s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/odk6mxi63epsq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #343
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/343/display/redirect?page=changes>
Changes:
[aldaircr] Change: Fixing typos on javadoc
------------------------------------------
[...truncated 279.79 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 04, 2020 6:47:45 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 04, 2020 6:47:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 04, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 04, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-96Dlnz62AKqzXvPKWe_0cA.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-H0wl-JrVfI14xKQoJGytZA.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-ag-yGEcUVWY3yvE1Lj4k4A.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-t9ErO4dwn6l6ZbGEz_WBiw.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-NDc71DIAUsQB_3H3wNoAjQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-4qruaQCltoSVlXLqr655VQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-eKe0tvcjDn9WG67iaHGAbA.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-EIEWcTqn_Nk-InpNKik1dg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-kg4UOG8Gqbxd7p5Q00EuDg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tbzw4hygQYPG33sECQGnVA.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-i7VKReI8XSaFuUPChvC9mg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-8C9FdiIYLZRsnT2Ypr0pug.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-PlRk8u95bibJMR8L5Z4XHg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-WwkQ_FKBAIW1UhsGkDaqCw.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-fOIWClYAZJ34ue0RrNEHRQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-VhwrwdqXxkMfqaemivKMGg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-rPQCoK_XBP3If5sE_krhNg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-4bYO-dZkJVgXvbk0Zdgs6g.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-TAZMZ2AtOh6F8WskqJwY9w.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-AwmLRCccP_ZPMPC5Cj-VWg.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-NNpgpsxjjP9XJCimQ4i9kA.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-ovph8tAl75iXacGnF1tQzQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-XaSNf8mq-YAgV-mZ15nFWQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-Yxwg4sX_SSbXh0Rtpcm1Ug.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-rb4VMaENZXebuCMNMtVFQQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-UIvP78hNNHMAExGsIPvVIQ.jar
Apr 04, 2020 6:47:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-nCbrGKs57GXZUwN3cA2oPQ.jar
Apr 04, 2020 6:47:48 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 1 seconds
Apr 04, 2020 6:47:48 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 04, 2020 6:47:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 04, 2020 6:47:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 04, 2020 6:47:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 04, 2020 6:47:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 04, 2020 6:47:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12075 bytes, hash MglF05D6ihKewj7A_Mtq_A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-MglF05D6ihKewj7A_Mtq_A.pb
Apr 04, 2020 6:47:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 04, 2020 6:47:50 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 04, 2020 6:47:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-03_23_47_50-12595263537052919804?project=apache-beam-testing
Apr 04, 2020 6:47:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-03_23_47_50-12595263537052919804
Apr 04, 2020 6:47:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-03_23_47_50-12595263537052919804
Apr 04, 2020 6:47:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T06:47:50.564Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 04, 2020 6:47:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:54.477Z: Checking permissions granted to controller Service Account.
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:58.956Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.518Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.641Z: Expanding GroupByKey operations into optimizable parts.
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.667Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.758Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.788Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.821Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 04, 2020 6:48:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:47:59.854Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 04, 2020 6:48:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:48:00.229Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 6:48:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:48:00.313Z: Starting 5 workers in us-central1-a...
Apr 04, 2020 6:48:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T06:48:12.693Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 04, 2020 6:48:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:48:24.062Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 6:48:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:48:42.038Z: Workers have started successfully.
Apr 04, 2020 6:48:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:48:42.073Z: Workers have started successfully.
Apr 04, 2020 6:49:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:49:17.348Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 6:49:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:49:17.565Z: Cleaning up.
Apr 04, 2020 6:49:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:49:17.673Z: Stopping worker pool...
Apr 04, 2020 6:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:50:53.399Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 04, 2020 6:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T06:50:53.448Z: Worker pool stopped.
Apr 04, 2020 6:51:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-03_23_47_50-12595263537052919804 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f9ee3586-da02-4e8c-bb8a-5f20bf109f58 and timestamp: 2020-04-04T06:51:00.146000000Z:
Metric: Value:
read_time 18.147
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 22.565 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 22s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/azwt4n3xtkip2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #342
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/342/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9577] Update container boot code to stage from dependencies, if
[rohde.samuel] Change delimeter to a dash as it is a reserved symbol in Windows
[github] Apply suggestions from code review
[robertwb] Use pointer recievers.
------------------------------------------
[...truncated 280.42 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 04, 2020 1:08:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 04, 2020 1:08:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 04, 2020 1:08:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 04, 2020 1:08:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tnfCV7zhgDWiat0cwAymmQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-ycloxwWjW1Qqkb0E-Pilaw.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-1ya6DCkaGq3azC00nPVi4g.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-bcAWEdXHRWEQq1jkcJTHyQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-PMAMfx5YzrUGUfB71t8XwQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-8UqFu8kGgKQK6lzFjRXv8w.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-52dqxR0tQ-hQp6E8I_j-jQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-5w_m93pT3sF7fqZkYFs4jQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-pOkzh7K20WOjIqxP1iTZTQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-FjYRla-C6BMlutaakeeW9g.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-1sMqWqWxMHCsBo5X-JPumQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-JG2jE3-xIRRx2qI87tnywg.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-RyN_fHSGYZ1EQqx4QOJbvg.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-9wArdKJf5ih3qsUNatw-QA.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-xHrM8axxTSFLVN9BOpGl2A.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-nyOQ9nHT6BAcdxAfgwoKzw.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-o1AbMVK48WiDWRSNICY7Yw.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-REAzKHiDpa5BimjB-iuP8Q.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-TcdXeFNR8gWm_2b6YWds-w.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-T2e-mFfIgvedoYrAEtxbJw.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-OdePDIoE9OQXoVEG1PRfNA.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-leOgZWRWNZwnsemHop3n4Q.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-Buf_ppw089lJeiBtMAEDAA.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Ue149Dfv4jRqmJsRcNjvOQ.jar
Apr 04, 2020 1:08:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-pYWCghGjb6QYxH2Jhvkt1A.jar
Apr 04, 2020 1:08:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-LkY0As_QkQVRCx_3JBV05A.jar
Apr 04, 2020 1:08:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-n9Jg1f_uBOFmN9CIktjQmA.jar
Apr 04, 2020 1:08:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 04, 2020 1:08:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash XY1PAUuUnZgJMRV6Hazq3A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XY1PAUuUnZgJMRV6Hazq3A.pb
Apr 04, 2020 1:08:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 04, 2020 1:08:41 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 04, 2020 1:08:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-03_18_08_41-7897629646224704050?project=apache-beam-testing
Apr 04, 2020 1:08:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-03_18_08_41-7897629646224704050
Apr 04, 2020 1:08:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-03_18_08_41-7897629646224704050
Apr 04, 2020 1:08:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T01:08:41.256Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 04, 2020 1:08:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:45.940Z: Checking permissions granted to controller Service Account.
Apr 04, 2020 1:08:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.053Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 04, 2020 1:08:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.644Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 04, 2020 1:08:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.720Z: Expanding GroupByKey operations into optimizable parts.
Apr 04, 2020 1:08:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.758Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.855Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.888Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.923Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:52.950Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:53.294Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 1:08:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:08:53.370Z: Starting 5 workers in us-central1-a...
Apr 04, 2020 1:09:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-04T01:09:07.103Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 04, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:09:17.457Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:09:17.487Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 04, 2020 1:09:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:09:22.971Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 04, 2020 1:09:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:09:37.331Z: Workers have started successfully.
Apr 04, 2020 1:09:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:09:37.368Z: Workers have started successfully.
Apr 04, 2020 1:10:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:10:18.573Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 04, 2020 1:10:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:10:18.813Z: Cleaning up.
Apr 04, 2020 1:10:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:10:18.898Z: Stopping worker pool...
Apr 04, 2020 1:12:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:12:07.409Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 04, 2020 1:12:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-04T01:12:07.468Z: Worker pool stopped.
Apr 04, 2020 1:12:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-03_18_08_41-7897629646224704050 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 53c5148f-1a94-435b-81ca-b1f62707edab and timestamp: 2020-04-04T01:12:15.351000000Z:
Metric: Value:
read_time 20.968
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 49.281 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 9s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/wyztsgecet4xs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #341
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/341/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-9686] Get default TmpCheckpointDir value from PipelineOptions
[spoorti] [BEAM-9660]: Add an explicit check for integer overflow.
[boyuanz] Populate source data from SDF
[boyuanz] Update Timer encoding
[mxm] [BEAM-9645] Fix premature removal of Docker container and logs
------------------------------------------
[...truncated 279.38 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 03, 2020 6:48:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 03, 2020 6:48:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 03, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 03, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 03, 2020 6:48:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-Zs4XowM-fVa6R9UllAnyCA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-cX34XvR5HwwyajrtHVIZiA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-SSidnHRzzyINkZ1uFgIDzQ.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-B6BonDTwxqSYIOVH1Yakyg.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-z2igtsaGt1XdGT6H7CadlA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-A7SMXGugoV5RLRoe8JEu-A.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-2ypqQ-t46XYHF-PIid_GXA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-mT11-camL5JXSMQR55-vYA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-TmGcT8vUVp6JUU0oSRrWzw.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-pQQVBhysNzJB3XWw1UvNhA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-Rds2abY8ZuVWeeirUqOVNQ.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-nvhfJ_H2hjE3V0uTzyd-SA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-fhx3vVFgf-lkTS_gYPcZnA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-UJ1U5KTff0DAiZzXh7X8og.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-upAeey5dw0DJrvUe-iPG2w.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-QXQN10u3qrNcR3zhibu3-g.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-X5c3C15szS4MuvQ-C7j7nA.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-4WdH2o7LHa3clVjunIzfZw.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-pBKSUHYu_VLD7lItItwGwg.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-u1pWTAbyPziTphfoVBPyRg.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-9mnSxXU-wmCSOU5RtOfX-g.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-pr4BFt5YclGh8ytazNcJXQ.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-t9b3HToF51ExD7_s34fVkg.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-CXdiiamjSSs7saUsJU-r4Q.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-ms_qXgY59MeO-wofBgT67A.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-SSWT5z2A7qI55hjw13UA9g.jar
Apr 03, 2020 6:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-B_piGV4BlwrW6yqJrMNLUg.jar
Apr 03, 2020 6:48:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash BpDGw66pcK7aeNKXWfgVdg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BpDGw66pcK7aeNKXWfgVdg.pb
Apr 03, 2020 6:48:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 03, 2020 6:48:42 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 03, 2020 6:48:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-03_11_48_42-6540985901245092348?project=apache-beam-testing
Apr 03, 2020 6:48:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-03_11_48_42-6540985901245092348
Apr 03, 2020 6:48:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-03_11_48_42-6540985901245092348
Apr 03, 2020 6:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T18:48:42.331Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 03, 2020 6:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:45.678Z: Checking permissions granted to controller Service Account.
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:48.957Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.533Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.573Z: Expanding GroupByKey operations into optimizable parts.
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.602Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.683Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.715Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.739Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 03, 2020 6:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:49.764Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 03, 2020 6:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:50.096Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 6:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:48:50.167Z: Starting 5 workers in us-central1-f...
Apr 03, 2020 6:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T18:48:57.067Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 03, 2020 6:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:49:17.594Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 6:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:49:17.621Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 03, 2020 6:49:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:49:23.148Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 6:49:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:49:37.217Z: Workers have started successfully.
Apr 03, 2020 6:49:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:49:37.252Z: Workers have started successfully.
Apr 03, 2020 6:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:50:20.073Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 6:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:50:20.307Z: Cleaning up.
Apr 03, 2020 6:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:50:20.421Z: Stopping worker pool...
Apr 03, 2020 6:52:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:52:24.659Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 03, 2020 6:52:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T18:52:25.069Z: Worker pool stopped.
Apr 03, 2020 6:52:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-03_11_48_42-6540985901245092348 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1f140708-9f95-4aac-85bc-2e482337f3b5 and timestamp: 2020-04-03T18:52:33.575000000Z:
Metric: Value:
read_time 24.072
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 4 mins 3.953 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 9s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/ebxm4scjhu5xy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #340
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/340/display/redirect?page=changes>
Changes:
[ameihm] [BEAM-9476] KinesisIO retry LimitExceededException
[alex] [BEAM-9044] Protobuf options to Schema options
------------------------------------------
[...truncated 278.88 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 03, 2020 12:53:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 12:53:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 12:53:08 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 12:53:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 12:53:08 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 03, 2020 12:53:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests--DDdkCdvJO92K633eqAJlg.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-IagixR3Aesez3p3c21vrnA.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-lIR3EjLD4jGf3B9QoR2DOw.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-q5p2h9wMy6FyHtWNQuO_ZQ.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-_Et2xSxqB9raW9hBaau7bg.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-mJu3L90wH9DxqiNChqCPpQ.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-ixcGjdVtDAiNnHDFoVhoXA.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-RQBpZ2hMhgUwhmJayMgwUg.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-YQAIGhYxkCVab78u364ISg.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-avD2DHoSvP1jZK6rEBuu6w.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-42PhDdXN3HC4FN4imeUs7w.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT--WrYQ1TcmyxmK125dHGd2Q.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-jpJ1Xb-A3TidpY6t3J7qgw.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-6dglKksn2t9Pnzj7gdQh0Q.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-kGrxz_c_rU5sXwORB2ZUNQ.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-Y7fIg_hWSRAxgE0LM1UJ9Q.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-Hh2Bh-cwYJYa0T5EpKgORg.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-n6i93XZAaefpmXrrLZ3_3w.jar
Apr 03, 2020 12:53:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-r8SxkNHVP1N2cK0smdmfyw.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-V79MPcxfGwpLyckrlXsRBw.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-St9InU0oyXDzzac1H6glvQ.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-vKl60GiUqPolNoKJPutxlA.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-rRYD8OYc2P_pEtDdsZRqzg.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-6iz7yX9YeQEnRve65G1Mcw.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-8zYYwr5XMEUikmGzY6YK4Q.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-HOe6NANOticE8jIrsYjAug.jar
Apr 03, 2020 12:53:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-4XBK_vTmnYmaZ-e4bds-3w.jar
Apr 03, 2020 12:53:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 03, 2020 12:53:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash vV44GH-Doex56txxLKxAYQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vV44GH-Doex56txxLKxAYQ.pb
Apr 03, 2020 12:53:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 03, 2020 12:53:14 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 03, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-03_05_53_14-4267105152473982969?project=apache-beam-testing
Apr 03, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-03_05_53_14-4267105152473982969
Apr 03, 2020 12:53:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-03_05_53_14-4267105152473982969
Apr 03, 2020 12:53:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T12:53:14.209Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 03, 2020 12:53:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:18.340Z: Checking permissions granted to controller Service Account.
Apr 03, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:21.824Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 03, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.410Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 03, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.472Z: Expanding GroupByKey operations into optimizable parts.
Apr 03, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.508Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.648Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.771Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.807Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:22.846Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:23.378Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 12:53:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:23.463Z: Starting 5 workers in us-central1-a...
Apr 03, 2020 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:53:54.109Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 12:53:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T12:53:56.639Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 03, 2020 12:54:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:54:17.546Z: Workers have started successfully.
Apr 03, 2020 12:54:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:54:17.584Z: Workers have started successfully.
Apr 03, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:54:56.978Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:54:57.212Z: Cleaning up.
Apr 03, 2020 12:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:54:57.323Z: Stopping worker pool...
Apr 03, 2020 12:56:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:56:17.325Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 03, 2020 12:56:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T12:56:17.395Z: Worker pool stopped.
Apr 03, 2020 12:56:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-03_05_53_14-4267105152473982969 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2ec8a4c6-f8fb-47d2-afaf-d3efdf162e0e and timestamp: 2020-04-03T12:56:24.472000000Z:
Metric: Value:
read_time 21.094
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.018 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 23.628 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/eqkxqqs25mef2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #339
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/339/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-9677] Fix path -> url typo in ArtifactUrlPayload
[github] [BEAM-9136]Add licenses for dependencies for Python (#11067)
------------------------------------------
[...truncated 279.85 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 03, 2020 6:45:42 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 03, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 03, 2020 6:45:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 03, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 03, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-jNUtTB9JmTAxUqCIqwRNOQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-N6CvFTC7BL5Es7L5U5gDGg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-VMUIP4hSQFB-uA88Iqtr8A.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-4c-8MxeNL87sB_qToOlOqQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded--vCWEbdCBBvzOmSNUcBjRw.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-ddAhQRzps7Nd8FJIn9J89Q.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-6AC0sMaOwLg0NU9-r-QrXQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-TZDBv3xq3Yi878UTA8ha8w.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-ladTZ8c-0s1tVgVYmkV4OQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-MSdM3PgeKj1NOtQoQdjfOQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-k8c-w5pR5fYv6GGdy9Aoiw.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-cI0ldyfnLrIC70dDgT3qQA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-Ffg1G3aP6P_btzBgjnTGdA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-V85H5hEzLjLZzMtoouudtA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-z_UplADdQ3-zpRLqM6v4zg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-kzYvPUH2oal6NS2pKJ4EnQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-pMzWddNpk8CDxV8eHNSQoA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-2oHT0FOaB9alP5l7uZ1Phg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-MjFcJNkIZBesdieOBXikBA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-Jr7utprTYfc6mHSLQWmKtg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-tXWFN4zlmvf5ElWe1wTBfA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-B1IaQ6MbJeslv0GmHcS1yQ.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-PNKvy8Cm1QzBW_l9ZCzw2w.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-olqs8qlsNc36mHbmn4HfnA.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-KoKLqo2FDImlriNfRu8vRg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-e_PBX-54O2IZ4uycsjHgCg.jar
Apr 03, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-XferS2ZJsapaTObe4RybWg.jar
Apr 03, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 03, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12082 bytes, hash Ao3Oadj1ZKhSoc71AosFSw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Ao3Oadj1ZKhSoc71AosFSw.pb
Apr 03, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 03, 2020 6:45:48 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 03, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-02_23_45_48-6881996285873258495?project=apache-beam-testing
Apr 03, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-02_23_45_48-6881996285873258495
Apr 03, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-02_23_45_48-6881996285873258495
Apr 03, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T06:45:48.547Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 03, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:52.242Z: Checking permissions granted to controller Service Account.
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:56.174Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:56.779Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:56.886Z: Expanding GroupByKey operations into optimizable parts.
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:56.916Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.002Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.033Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.069Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.103Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.457Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:45:57.541Z: Starting 5 workers in us-central1-f...
Apr 03, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T06:46:17.240Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 03, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:46:24.575Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 6:46:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:46:24.601Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 03, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:46:30.011Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:46:48.470Z: Workers have started successfully.
Apr 03, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:46:48.496Z: Workers have started successfully.
Apr 03, 2020 6:47:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:47:23.725Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 6:47:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:47:23.926Z: Cleaning up.
Apr 03, 2020 6:47:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:47:24.000Z: Stopping worker pool...
Apr 03, 2020 6:49:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:49:04.860Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 03, 2020 6:49:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T06:49:04.910Z: Worker pool stopped.
Apr 03, 2020 6:49:11 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-02_23_45_48-6881996285873258495 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d4e26e84-5c2f-4449-a081-1fedc3b1830d and timestamp: 2020-04-03T06:49:11.928000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.061
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 37.211 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 53s
101 actionable tasks: 70 executed, 31 from cache
Publishing build scan...
https://gradle.com/s/hkwzrgl55onlm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #338
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/338/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9577] Add dependency information to provision info.
[robertwb] Update go protos.
[lcwik] [BEAM-9562] Update missed TimerSpec conversion in Go SDK
[github] [BEAM-9667] Allow metrics in DoFn Setup (#11287)
[github] [BEAM-9624] Adds Convert to Accumulators operator for use in combiner
[github] Fix minor typo
[github] Fix minor typo
[github] Merge pull request #11290: [BEAM-9670] Fix nullability widening in
[github] [BEAM-4374] Short IDs for the Python SDK (#11286)
------------------------------------------
[...truncated 280.09 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 03, 2020 1:14:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 1:14:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 1:14:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 03, 2020 1:14:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 03, 2020 1:14:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-dGuMQ1sibYiym7cLUE8PmQ.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-qfmCho6e2gomTGJex8GP5w.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-Yd_zwhOH92jrHpKX4lzRRA.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-d20HCvHLi1mcy9_1ktIxSQ.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-f6NZyfZxJOGSq7NEWjpVqw.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-vKkx2tocv8ZZOkUK22CqQA.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-mxP0Gn6DTVN7HSEs2DV9Vg.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-QT76gcGmzOhM8227ig313w.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-89Y5tTPtbmhdyW_GavOBKA.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-9n3DgYUrCJg7qxlKUIJdog.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-dErCq_u8l-aX5daOIZB3PA.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-W8LACaaFFdft9OASbzonPQ.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-qC7K4YUOeiNs237d-SyEXQ.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-uShn8xJtzGYoVFnmiV1PRg.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-jQ2WCsyIGI5JCHOB7W6Dcw.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-rkrQ69RNtuse9fz3ePzGJA.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-PTyOyR2x-FNG_AvNRRMTYw.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-N54I8E9w8kB4s47WbDx73Q.jar
Apr 03, 2020 1:14:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-EWputU16f4GT_CufusYoWQ.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-cfk7GrXO4W34H4mibdW1hg.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-Gz-lcn2gZRq1qnwQGIUt3Q.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-omczHPQogCD5AoXYUvuB7g.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-T0mng9X_7KTr_nH9fos1mg.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-YfHuB02df9NrHlgyZL33rw.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-q8o_kNZjQGicu_scoMl3yg.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-Fs4XRu32e1YmEFWk23Norg.jar
Apr 03, 2020 1:14:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-cdzYpOuqZvuEDa4-WO8Y5w.jar
Apr 03, 2020 1:14:28 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 03, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 03, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 03, 2020 1:14:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 03, 2020 1:14:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 03, 2020 1:14:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 03, 2020 1:14:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash ieXWYV1NYG1cIQs2vnW55g> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ieXWYV1NYG1cIQs2vnW55g.pb
Apr 03, 2020 1:14:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 03, 2020 1:14:30 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 03, 2020 1:14:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-02_18_14_31-4661028773533775421?project=apache-beam-testing
Apr 03, 2020 1:14:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-02_18_14_31-4661028773533775421
Apr 03, 2020 1:14:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-02_18_14_31-4661028773533775421
Apr 03, 2020 1:14:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T01:14:31.053Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 03, 2020 1:14:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:36.732Z: Checking permissions granted to controller Service Account.
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:42.655Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.547Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.598Z: Expanding GroupByKey operations into optimizable parts.
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.629Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.727Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.767Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.814Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 03, 2020 1:14:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:43.858Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 03, 2020 1:14:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:44.443Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 1:14:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:14:44.521Z: Starting 5 workers in us-central1-a...
Apr 03, 2020 1:14:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-03T01:14:54.571Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 03, 2020 1:15:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:15:11.744Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 1:15:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:15:11.780Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 03, 2020 1:15:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:15:17.185Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 03, 2020 1:15:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:15:29.957Z: Workers have started successfully.
Apr 03, 2020 1:15:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:15:30.030Z: Workers have started successfully.
Apr 03, 2020 1:16:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:16:12.748Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 03, 2020 1:16:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:16:13.051Z: Cleaning up.
Apr 03, 2020 1:16:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:16:13.169Z: Stopping worker pool...
Apr 03, 2020 1:18:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:18:15.183Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 03, 2020 1:18:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-03T01:18:15.244Z: Worker pool stopped.
Apr 03, 2020 1:18:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-02_18_14_31-4661028773533775421 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ab90173b-fd26-4768-893d-f16e79568fcb and timestamp: 2020-04-03T01:18:25.246000000Z:
Metric: Value:
read_time 23.191
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 8.147 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 21s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/6b6somczwadpo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #337
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/337/display/redirect?page=changes>
Changes:
[valentyn] Fix a Py2/3 incompatibility in profiler.
[github] [BEAM-7923] An indicator of progress in notebooks (#11276)
------------------------------------------
[...truncated 279.70 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 02, 2020 7:13:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 7:13:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 7:13:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 7:13:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 7:13:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 02, 2020 7:13:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-jp1rf2EOI9xiE3lQEHXJ2A.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-8P4CpRuYSQai-SpxVLo6SA.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-0roJubfPzIg7jFTrToMcjw.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-lYjnecS0IvVAtYQcLKYtpw.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Na_udBtoDGAG5CRxVE0r1g.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-3pfEpccb6H57ipcNPhzd3w.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-dgBiHk68kMD4g5GBcen-5Q.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-PhlkNdnzXa_BQPZ83JiCVQ.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-NYqjRy350R3VhjNIosyhqg.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-5uAUN4Aq_MEoRSZ0xlv6bw.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-mPUUx1rEkOSTplOoIkEMxw.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-2tqY1dN4brMrznMa0ZOzOA.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-bQ4P-ykBitrZYJHEdzrwqw.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-MFBnukMH5_XXZ1cXde4kVA.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-xMx6j-bjBxY7kG6T7Q6nrQ.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-II9994D1JhEzgbGiJ-bQQQ.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-yQwkz4iaDX7Trl5BXHHx-g.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-zyTBKdhKLy5BPkbA_vkD8A.jar
Apr 02, 2020 7:13:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-HlGLjotEJCgfJW_qlhF-QA.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-hHHLIcmIfDpCfR14sqpbKQ.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-4M2KE5CZxAM6ztXj6QGIeg.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-MgBovU9hBLu812IhCgeOkA.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-oxa73JMlt64LgxvKdRgQEQ.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-DM0lbFNFJt26UjHCsZiZAQ.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-I6Axg2TdWDxdfxzIVwK82w.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-5saOldwdkKqI67u6bd0E2w.jar
Apr 02, 2020 7:13:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-bSph14XTSC-Nqcerde2FTQ.jar
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 02, 2020 7:13:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 02, 2020 7:13:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash lVTlOXuuF--gtggWE2Ek3A> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lVTlOXuuF--gtggWE2Ek3A.pb
Apr 02, 2020 7:13:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 02, 2020 7:13:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 02, 2020 7:13:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-02_12_13_22-14566807497559870145?project=apache-beam-testing
Apr 02, 2020 7:13:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-02_12_13_22-14566807497559870145
Apr 02, 2020 7:13:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-02_12_13_22-14566807497559870145
Apr 02, 2020 7:13:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T19:13:22.858Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 02, 2020 7:13:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:29.767Z: Checking permissions granted to controller Service Account.
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:38.839Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.491Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.544Z: Expanding GroupByKey operations into optimizable parts.
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.620Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.884Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.930Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:39.969Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 02, 2020 7:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:40.017Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 02, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:40.393Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:13:40.460Z: Starting 5 workers in us-central1-a...
Apr 02, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T19:13:40.618Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 02, 2020 7:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:14:13.862Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 7:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:14:13.902Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Apr 02, 2020 7:14:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:14:19.330Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 7:14:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:14:35.465Z: Workers have started successfully.
Apr 02, 2020 7:14:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:14:35.516Z: Workers have started successfully.
Apr 02, 2020 7:15:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:15:15.921Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 7:15:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:15:16.161Z: Cleaning up.
Apr 02, 2020 7:15:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:15:16.365Z: Stopping worker pool...
Apr 02, 2020 7:17:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:17:12.128Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 02, 2020 7:17:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T19:17:12.172Z: Worker pool stopped.
Apr 02, 2020 7:17:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-02_12_13_22-14566807497559870145 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
Load test results for test (ID): b27bd834-9541-4936-a911-ac162dbc3f36 and timestamp: 2020-04-02T19:17:19.026000000Z:
Metric: Value:
read_time 23.484
fields_read 4375276.0
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 4 mins 8.739 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 8s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/pvxvae4zdyins
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #336
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/336/display/redirect>
Changes:
------------------------------------------
[...truncated 280.49 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 02, 2020 12:49:40 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 02, 2020 12:49:41 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 02, 2020 12:49:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 02, 2020 12:49:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 02, 2020 12:49:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-9DDSWAZ_iracUTpXzF-5yQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-eSDmCwKm3zTvan0aRyt7KQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-2hYu1OlM4Sn43sEmeXAX8Q.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-JhEKK5EEVoEdShgZwB2NEw.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-fkM3xic4DjSEY7Fap3VbrA.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-yOu2MAMd-plM9mLnfazPSQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-Ze6ttaXECuGASTzfqDmsCg.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-B2n8Wdxx5Lbcdz79YKQFcw.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT--vzuxdkk-cq0Dj17fpFb7w.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-BXA_t2IL_2draxaUo_RsqQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-q_VNg0HIE6IED_VqeSIGMg.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-0B45-8E5Ud-8DYjhXzxWmA.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-81-fOlyTD_yOZ6P5n5jgAw.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-5PI0JElgBWvqRf39M07ABA.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-oWO3XNj7v1OLRYDbq2NGOQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-NHHc3sz_pXcODp76rGXudg.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-M27MkQ4XT3bt-rrFU648BA.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-rX_2kVaKVGVJudrn63QVsg.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-PbDqN0bCwUN-V2mqp1QUoQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-YKAv32WN3Azb8kw7fqVBmQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-bz5fAcgraDi65AQOVHrpgQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-Km7C1Bq0c-_UvYR_ohU8kQ.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-TkOvE5wXaIxlGyH-LB1Akw.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-bfT800mA9SVNYmy7rDC-2g.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-VQBJSSvSRRIDxIiSk8-zRw.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-TN8xEHGXGMsAADhI5jFXVA.jar
Apr 02, 2020 12:49:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-8LhddTIbzOlPuyD06Od_NQ.jar
Apr 02, 2020 12:49:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 4 seconds
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 02, 2020 12:49:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash VPSf5RVBy_QNDN4y9Un6Bg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-VPSf5RVBy_QNDN4y9Un6Bg.pb
Apr 02, 2020 12:49:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 02, 2020 12:49:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 02, 2020 12:49:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-02_05_49_48-3500525741815209212?project=apache-beam-testing
Apr 02, 2020 12:49:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-02_05_49_48-3500525741815209212
Apr 02, 2020 12:49:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-02_05_49_48-3500525741815209212
Apr 02, 2020 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T12:49:48.690Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 02, 2020 12:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:52.205Z: Checking permissions granted to controller Service Account.
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:57.658Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.362Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.400Z: Expanding GroupByKey operations into optimizable parts.
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.428Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.560Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.595Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.623Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 02, 2020 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:58.661Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 02, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:59.007Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:49:59.089Z: Starting 5 workers in us-central1-f...
Apr 02, 2020 12:50:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T12:50:20.284Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 02, 2020 12:50:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:50:23.779Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:50:41.548Z: Workers have started successfully.
Apr 02, 2020 12:50:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:50:41.583Z: Workers have started successfully.
Apr 02, 2020 12:51:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:51:21.311Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 12:51:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:51:21.626Z: Cleaning up.
Apr 02, 2020 12:51:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:51:21.711Z: Stopping worker pool...
Apr 02, 2020 12:53:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:53:39.846Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 02, 2020 12:53:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T12:53:39.932Z: Worker pool stopped.
Apr 02, 2020 12:53:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-02_05_49_48-3500525741815209212 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7eee6140-3aeb-4b9d-b152-e74b1ee51a04 and timestamp: 2020-04-02T12:53:46.725000000Z:
Metric: Value:
fields_read 4375276.0
read_time 22.432
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 14.401 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 25s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/czj7tjqpxbqx2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #335
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/335/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-9668] Disable tests till Dataflow containers are updated.
[github] [BEAM-9652] Ensure that the multipartition write sets the correct coder
[github] [BEAM-8889]add experiment flag use_grpc_for_gcs (#11183)
------------------------------------------
[...truncated 281.47 KB...]
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 02, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 6:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 6:45:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 6:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 6:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 02, 2020 6:45:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-uBmu2Yb-uY9dRwYWojkhNw.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-y_KgY98Q-SkmsxzRHqSMKg.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-W8Rv-flOH0F79ee3qCjFew.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-BPrpWCi4DLBGnpwzeTDjow.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-NIYrJV6Mf9yS8EQmPke8fg.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-CZCqlzC3_YJPaKmasnWopA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-q7EWwwGfGwSZ4a27VRLeRw.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-9msV6IRxlNI9lHiVtTJ77Q.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-WuHd7OPcaJv76I9HyfbyeA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-yVKH2kQu0IcdOscGR3mA3g.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-f9mi2HQKUzg9gUlA9ig_yA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-uJDAc0R-8gGvE58QYU1vGQ.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-_iQaL5KB5TdW6PhWERaSuA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-6PA_sl8PDYCxMW3v3ahJeA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-2-ZQ_oU8VIVSZg3aQC0ccg.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-0teJG0i7uVZAkKz8SRxWAw.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.cloud/google-cloud-spanner/1.49.1/19dfd15e889a41c306b5dd62a622db6c471cb6fc/google-cloud-spanner-1.49.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-spanner-1.49.1-nxwqdDjvXeAUxVwwqjuHbA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-BvCRr8ylzrC-KGRQy646xA.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-XEgyHzFLIpk5b2MbXQ6T_g.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.api.grpc/proto-google-cloud-spanner-admin-database-v1/1.49.1/b1c18dc081a1858060577767af00dec47f0ddeeb/proto-google-cloud-spanner-admin-database-v1-1.49.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-1.49.1-SVE4dwmTzXSAEXJWTDRlOg.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.api.grpc/proto-google-cloud-spanner-admin-instance-v1/1.49.1/d0e502423bfed8985952dcb2f909cb8417b6923e/proto-google-cloud-spanner-admin-instance-v1-1.49.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-1.49.1-u98qw01DBrtridvy0P1BCw.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-W66muNkEGC0PuI7255RpLw.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.api.grpc/proto-google-cloud-spanner-v1/1.49.1/d9d02615204a606caf87c91f67c0ff9c31e1ba52/proto-google-cloud-spanner-v1-1.49.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-1.49.1-LOX58cHoNzck-HGvlutpVQ.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-7nFMcXah-CN7KZPvso25Mg.jar
Apr 02, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-ryIdlypT6XCycMi23tr7WA.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-J-Nih7nyjFi5Gd4ExVi4Aw.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-K-pKABejAkYGjwxZG4x6ZQ.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.threeten/threetenbp/1.4.1/8e0de17c5077523503812837fa069459103fb714/threetenbp-1.4.1.jar to gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.4.1-nlDPIP6A5wjxKspkiqVDXQ.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-UtXEH6wzhyAOYWVy4qJacQ.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-S7Zoob-JUpV_ApsUcEdycg.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-yp2hB2NW7l-E5LgdZzl2RA.jar
Apr 02, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-dseTb_4zW2T2g_oDiT7u6A.jar
Apr 02, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 173 files cached, 32 files newly uploaded in 2 seconds
Apr 02, 2020 6:45:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash kUiEEGK6ALfx8YQ_u04kbg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kUiEEGK6ALfx8YQ_u04kbg.pb
Apr 02, 2020 6:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 02, 2020 6:45:26 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 02, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-01_23_45_26-1733321028448184023?project=apache-beam-testing
Apr 02, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-01_23_45_26-1733321028448184023
Apr 02, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-01_23_45_26-1733321028448184023
Apr 02, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T06:45:26.158Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 02, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:29.743Z: Checking permissions granted to controller Service Account.
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:35.403Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.046Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.084Z: Expanding GroupByKey operations into optimizable parts.
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.117Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.194Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.223Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.258Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.292Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.656Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:45:36.725Z: Starting 5 workers in us-central1-f...
Apr 02, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T06:45:41.099Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 02, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:46:05.021Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 6:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:46:05.045Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 02, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:46:10.479Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:46:23.899Z: Workers have started successfully.
Apr 02, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:46:23.934Z: Workers have started successfully.
Apr 02, 2020 6:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:47:00.150Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 6:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:47:00.377Z: Cleaning up.
Apr 02, 2020 6:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:47:00.455Z: Stopping worker pool...
Apr 02, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:48:39.353Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 02, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T06:48:39.403Z: Worker pool stopped.
Apr 02, 2020 6:48:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-01_23_45_26-1733321028448184023 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 98eebb77-2d9f-49c2-9bfa-2d902c624698 and timestamp: 2020-04-02T06:48:45.912000000Z:
Metric: Value:
read_time 19.697
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 32.657 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/zqjnvi6yuowiu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #334
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/334/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9340] Populate requirement for timer families.
[pabloem] Revert "Merge pull request #11104 from y1chi/update_tornado_test"
[daniel.o.programmer] [BEAM-9642] Create runtime invokers for SDF methods.
------------------------------------------
[...truncated 279.28 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 02, 2020 1:11:00 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 02, 2020 1:11:01 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 02, 2020 1:11:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 02, 2020 1:11:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 02, 2020 1:11:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-4Q9Oo0jmStzuBHdamOqrJw.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-E2IYwlLzl-3iKpkBBRPMQA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-ccTG61QrWBuDXlF1rxhYmA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-B55kT8nrF625ITAB8L8Fvg.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-tVUGlV2TQ9bjeJd7dbaHQQ.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-swgozZqepoISLJE3erXobA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-DJxNeN--f8JQxBHRoZylsg.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-EHe9fNFEH9CFFvsroYrmRg.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-mWD-harklyKLAm8PQjn8XA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-1rXiiZVAvYByfADkjgaTkA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-00EijxsT7iyyqAeZRR5lJQ.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-or3ffd9koFvFWzCzqtTgcA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-55pB7-5LQq8jwgISkwmhCA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-uY4rp3qhdNFkfMO3vE5c_w.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-9bjTdcVcSDnWqCrsVoQJhw.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-ZfPnPvcAvNMJaSGocp9KXA.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-KHlhrj2ZJKiW54uW0ia4Vw.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-2RVPVcsXwyS1bbUT8hV72A.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-p4yXMXwH1MwK09_aoQ8tUQ.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-IYIBCKGWc6qdc-fIzxiXug.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-rU2ZZDWdUL3FxZomx0Vbig.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-4o8dy9M7ST9ledu1Wk_HkQ.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-_pl1-R50JB9dAnAcRAFE0w.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-aSsX1w0ovkvhcRHYLiTS_Q.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-0wMcVLz3OrI78PalES5cnw.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-l1AbkzNR4KQPOnM38AcIkg.jar
Apr 02, 2020 1:11:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-XlN1t_Fk01njBieCkumNIw.jar
Apr 02, 2020 1:11:03 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash tnUK35e60HDRgDfbQbnaSg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-tnUK35e60HDRgDfbQbnaSg.pb
Apr 02, 2020 1:11:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 02, 2020 1:11:05 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 02, 2020 1:11:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-01_18_11_05-10148155721789860358?project=apache-beam-testing
Apr 02, 2020 1:11:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-01_18_11_05-10148155721789860358
Apr 02, 2020 1:11:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-01_18_11_05-10148155721789860358
Apr 02, 2020 1:11:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T01:11:05.318Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 02, 2020 1:11:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:09.903Z: Checking permissions granted to controller Service Account.
Apr 02, 2020 1:11:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:13.411Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.212Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.306Z: Expanding GroupByKey operations into optimizable parts.
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.342Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.425Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.453Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.482Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:14.524Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:15.079Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 1:11:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:15.148Z: Starting 5 workers in us-central1-f...
Apr 02, 2020 1:11:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-02T01:11:29.346Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 02, 2020 1:11:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:38.442Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 1:11:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:38.476Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 02, 2020 1:11:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:43.996Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 02, 2020 1:11:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:57.986Z: Workers have started successfully.
Apr 02, 2020 1:11:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:11:58.020Z: Workers have started successfully.
Apr 02, 2020 1:12:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:12:44.420Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 02, 2020 1:12:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:12:44.638Z: Cleaning up.
Apr 02, 2020 1:12:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:12:44.751Z: Stopping worker pool...
Apr 02, 2020 1:14:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:14:37.278Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 02, 2020 1:14:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-02T01:14:37.344Z: Worker pool stopped.
Apr 02, 2020 1:14:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-01_18_11_05-10148155721789860358 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3ba3bac2-65de-4215-bcdd-2c2ae501c131 and timestamp: 2020-04-02T01:14:42.924000000Z:
Metric: Value:
fields_read 4375276.0
read_time 27.275
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 50.276 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/3uuyquop77ark
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #333
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/333/display/redirect?page=changes>
Changes:
[zyichi] Update BigQuery source in bigquery_tornadoes example
[rohde.samuel] Add dependency comment in streaming cache
[ehudm] [BEAM-1894] Remove obsolete EagerRunner test
------------------------------------------
[...truncated 296.11 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 01, 2020 7:13:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 01, 2020 7:13:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 01, 2020 7:13:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 01, 2020 7:13:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-vVs_AONeqnKDWt0N6u_ckw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-0E6dLNs6PNU2BO-mtoJn8A.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-r-O-773hPF5fv60cuWtzkQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-IbdIaX2oim6ha4oUdCkFxw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-u0Kjiuvw1RL2H9mNPoCptg.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT--dU_llJUMi0kNunMln7e-A.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-UXLe_OcZMnzgXGtYzOrrkA.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-Ztcl7rTOzF2uXCcyECqBYQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-2El27WJvSdaBtwT3yBUfbQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-ZMHwd8iLBp1y0KD1Ixd85A.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-nsuALs4kSTEOzsXdiG3sug.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bN1eQYbSk5c_62Z06Z-Oug.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-lRTk3y5sdgVfAJ0BfkFFoQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-erwGzV9-tTKJwWmPAdYIsQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-Xhh_D1oZhsmR-8Rqk04piw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-A0TfXdELkZFhlL05z4pEuw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-5y-UJv5r0RPShR9QuQ3jrw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-ElMGU2k0hIy6hehgRfLgMg.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-dX99Uo6Rnj1hJF2CcJKvOA.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-58jy_hzr4IB08NI4wlqf6g.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-Wx47GhphIYki1ucpkirzRg.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-j6yfi__iZ8_l-n0rmcO4uw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-5EhyZGYEIMxKRLqcl1tPeQ.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-u9fE742ncDAwPw87UUqPOw.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-2tRiMz5xecLy2r5T7BQmMg.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-2Rf8hy_qXJsFNEJstwGHaA.jar
Apr 01, 2020 7:13:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-qVGsen4CPwbzRVrBaRCwAA.jar
Apr 01, 2020 7:13:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 01, 2020 7:13:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12085 bytes, hash wTnKGXc3nP9WaGgVhGj2Rw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-wTnKGXc3nP9WaGgVhGj2Rw.pb
Apr 01, 2020 7:13:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 01, 2020 7:13:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 01, 2020 7:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-01_12_13_31-5874266845699615778?project=apache-beam-testing
Apr 01, 2020 7:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-01_12_13_31-5874266845699615778
Apr 01, 2020 7:13:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-01_12_13_31-5874266845699615778
Apr 01, 2020 7:13:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T19:13:31.553Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 01, 2020 7:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:35.271Z: Checking permissions granted to controller Service Account.
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:41.474Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.187Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.226Z: Expanding GroupByKey operations into optimizable parts.
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.260Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.367Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.411Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.455Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 01, 2020 7:13:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.500Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 01, 2020 7:13:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:42.990Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 7:13:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:13:43.081Z: Starting 5 workers in us-central1-f...
Apr 01, 2020 7:14:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:14:06.974Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 7:14:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:14:07.013Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Apr 01, 2020 7:14:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:14:12.424Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 7:14:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T19:14:13.581Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 01, 2020 7:14:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:14:25.926Z: Workers have started successfully.
Apr 01, 2020 7:14:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:14:25.967Z: Workers have started successfully.
Apr 01, 2020 7:15:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:15:03.903Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 7:15:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:15:04.172Z: Cleaning up.
Apr 01, 2020 7:15:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:15:04.259Z: Stopping worker pool...
Apr 01, 2020 7:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:17:11.253Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 01, 2020 7:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T19:17:11.304Z: Worker pool stopped.
Apr 01, 2020 7:17:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-01_12_13_31-5874266845699615778 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ef666165-34b2-4b28-a91d-2d4f4994fcd4 and timestamp: 2020-04-01T19:17:18.486000000Z:
Metric: Value:
fields_read 4375276.0
read_time 21.895
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 4 mins 0.356 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 10s
101 actionable tasks: 73 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/2kwaaoct3e66i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #332
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/332/display/redirect?page=changes>
Changes:
[jozsi] Update Jet version to 4.0
[jozsi] Update Jet Runner web page with info about 4.0
[jozsi] Add Beam-Jet compatibility table
------------------------------------------
[...truncated 280.31 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 01, 2020 12:59:53 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 01, 2020 12:59:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 01, 2020 12:59:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 01, 2020 12:59:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-KQfx4b_zbIN_y3f0NbJekA.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-hdFFZqu1KLhUQFntWotpNQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-diwHv53CEBW-YF-I1X7yaw.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT--s6hjrpfmbb0V_Z7NCDj8Q.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-vTb6prTj3M4tKd793Ixz0Q.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-KCEmsIzx0f-F5ANH5e45xg.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-rbjy06CaGH5Hj8gOa-0Sgg.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-HuwinnqJvi7zKO0vX3d2KA.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-lrJvJiDuTlymnjSxNZDfxQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-nXFiCkf_dG5QL1znXBjp6Q.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-2L87cLwpU9wCvJC84tq7uw.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-cqOqHsiIGjWOrsHQZCn8Aw.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-lmecZ5WqlFQ0UP5WV-XTnQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-7fyOYnb1-w885EI9OA8i8w.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-CR2JiqBwXx3z1UKDoXWT_Q.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-OqHJVSLF-thQZGdpUjhDNg.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-cuN4EZivf215zGQIYaLLlQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-pi2c63zDhqgeDl7B_-6g1A.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-0cCrFlHDte1NqXP6mRV_iQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-_NsJPqMgOpjwYEjkMM3Ejw.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-SkMIxNg9Oek_0aPIBEBkXQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-wIC74cx0gMQSRC9tAaxMhg.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-NN_rNTxmhifoPXxlaHtIlA.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-JJDXoX1Xc6vdUcR8_rEE1Q.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-DJMUNvTf-59mKMqqdPm4Ng.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-A_wJijSokgyt3jeDLOYSQQ.jar
Apr 01, 2020 12:59:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-VGSX-g6tafysM_U583uCvQ.jar
Apr 01, 2020 12:59:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 01, 2020 12:59:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 52KQke5p27ejhU1ZxcWtaQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-52KQke5p27ejhU1ZxcWtaQ.pb
Apr 01, 2020 12:59:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 01, 2020 12:59:58 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 01, 2020 12:59:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-04-01_05_59_58-2083235987100629882?project=apache-beam-testing
Apr 01, 2020 12:59:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-04-01_05_59_58-2083235987100629882
Apr 01, 2020 12:59:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-04-01_05_59_58-2083235987100629882
Apr 01, 2020 1:00:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T12:59:58.493Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 01, 2020 1:00:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:01.734Z: Checking permissions granted to controller Service Account.
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.078Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.645Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.684Z: Expanding GroupByKey operations into optimizable parts.
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.705Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.772Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.803Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.829Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 01, 2020 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:05.858Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 01, 2020 1:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:06.259Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 1:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:06.319Z: Starting 5 workers in us-central1-f...
Apr 01, 2020 1:00:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T13:00:15.444Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 01, 2020 1:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:00:34.832Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 1:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:01:01.016Z: Workers have started successfully.
Apr 01, 2020 1:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:01:01.054Z: Workers have started successfully.
Apr 01, 2020 1:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:01:39.080Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 1:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:01:39.297Z: Cleaning up.
Apr 01, 2020 1:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:01:39.377Z: Stopping worker pool...
Apr 01, 2020 1:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:03:42.042Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 01, 2020 1:03:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T13:03:42.083Z: Worker pool stopped.
Apr 01, 2020 1:03:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-04-01_05_59_58-2083235987100629882 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3bceff3e-15c4-4659-b9f8-6046b0b6071c and timestamp: 2020-04-01T13:03:47.850000000Z:
Metric: Value:
read_time 19.616
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 4 mins 3.04 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 22s
101 actionable tasks: 72 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/odpm52d4xvqwo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #331
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/331/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-9263] Bump up python sdk dataflow environment major versions
[pabloem] [BEAM-9608] Increase reliance on Context Managers for FnApiRunner
------------------------------------------
[...truncated 279.65 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 01, 2020 6:57:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 6:57:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 6:57:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 6:57:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 6:57:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 01, 2020 6:57:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-DL8THinSEhQMT1Vmjpyjxg.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-O8crVsrNt0HqxdA_przceg.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-4_t0sOJqi23xQPMIQO0RXw.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-13h6QqtV_zD2TY4h_YEgWw.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-38MVaU48GU5aDLmF5_jyjA.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-SyIPdf-POgrbgEb5taF8yA.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-il9Sss6DDoV2LX8Vb1R1qw.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-y9YfpfkXh-olm20kYdfSZw.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-y7ysKj5CAjt8T6vcCBn0BA.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-gb0VD4iP9Tg5pMBSQ9sw7w.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-6jASDD0U9mdmjXajGrdWgg.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-hR_S8-jQR8z9rhWFeQ4xvQ.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-MwLXrc4gwyqD1RTCQM8-AQ.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-2mrLfXSKIU6tAYk1i1gocw.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-4LTe46ty5S-fnX1fLTSVHQ.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-NUxQdvY4SrmynvFoyC_Y-w.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-23z94qmcEjneEeIZGKArSg.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-XBk8BRHaISO6N6gGJgqtLg.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-RFKZxhS-UBg4ZTXAfRL59Q.jar
Apr 01, 2020 6:57:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-FKCERC-vSeI-RNp97XfQaA.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-y5LRwnc2mW_LMRVi3wSZxQ.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-uO7LLKep6JyXWx7IvuF8oA.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-yrJkrfd7GViBoueN_ae6ig.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests--5m5NEk-uXetl54CBHRyuA.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-p9HqvuL0mU6lDBEQLRSCuw.jar
Apr 01, 2020 6:57:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-L0RYg4_odFCKWXglVEy46Q.jar
Apr 01, 2020 6:57:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-Xnhex4SWK0QNjmqeC-ozOw.jar
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 22 seconds
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 01, 2020 6:57:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 01, 2020 6:57:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 2YggM6t__4DFHyGI_oqYFA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2YggM6t__4DFHyGI_oqYFA.pb
Apr 01, 2020 6:57:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 01, 2020 6:57:54 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 01, 2020 6:57:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-31_23_57_54-515331815103422319?project=apache-beam-testing
Apr 01, 2020 6:57:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-31_23_57_54-515331815103422319
Apr 01, 2020 6:57:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-31_23_57_54-515331815103422319
Apr 01, 2020 6:57:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T06:57:54.903Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 01, 2020 6:57:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:57:58.151Z: Checking permissions granted to controller Service Account.
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:02.776Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.448Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.495Z: Expanding GroupByKey operations into optimizable parts.
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.539Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.621Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.659Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.698Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 01, 2020 6:58:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:03.743Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 01, 2020 6:58:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:04.219Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 6:58:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:04.320Z: Starting 5 workers in us-central1-f...
Apr 01, 2020 6:58:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T06:58:28.515Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 01, 2020 6:58:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:29.451Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 6:58:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:29.539Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Apr 01, 2020 6:58:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:35.085Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 6:58:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:53.164Z: Workers have started successfully.
Apr 01, 2020 6:58:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:58:53.202Z: Workers have started successfully.
Apr 01, 2020 6:59:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:59:34.780Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 6:59:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:59:34.959Z: Cleaning up.
Apr 01, 2020 6:59:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T06:59:35.030Z: Stopping worker pool...
Apr 01, 2020 7:01:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T07:01:31.163Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 01, 2020 7:01:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T07:01:31.217Z: Worker pool stopped.
Apr 01, 2020 7:01:38 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-31_23_57_54-515331815103422319 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 64ec91a8-45d5-4abe-91ac-86d5ed75fc22 and timestamp: 2020-04-01T07:01:38.820000000Z:
Metric: Value:
fields_read 4375276.0
read_time 24.47
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 17.419 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 27s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/zlp7vl3wvgkys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #330
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/330/display/redirect?page=changes>
Changes:
[kyoungha] [BEAM-9325] Added Proper Write Method in UnownedOutputStream
[kyoungha] fix format warning
[samuelw] [BEAM-9399] Change the redirection of System.err to be a custom
[kyoungha] [BEAM-9325] reflect comment : inline testing methods
[robertwb] [BEAM-9577] Rename the Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Define the new Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Regenerate protos.
[robertwb] [BEAM-9577] Implement the new Artifact{Staging,Retrieval}Services in
[samuelw] Fix missing test import
[kyoungha] [BEAM-9325] reflect comment : Fix JAXBCoder + change test
[chamikara] Refactors X-Lang test pipelines.
------------------------------------------
[...truncated 282.97 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Apr 01, 2020 1:49:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Apr 01, 2020 1:49:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Apr 01, 2020 1:49:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-0zk1ktalCWNnVJaEZci1bw.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-Zq3hAgMcC629CYiMnQcBvw.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-YK5ESowkJJUxA2pKXpEBpQ.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-vj-4Qwespc3ByMJWb9kWAg.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-S0hZ0XhRkCfkRIySiguK-A.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-FI4PxlZe7KASj7tyE05rWA.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-Oy7J26gRPk3xTJQGtcCg0A.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-AP0xdQksnglp0AiUDvWgkw.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-bUP2hZriMcBKtkik8RMhIg.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-Ywp7wkrnsetaV1WbdAQ5mQ.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-AjtZ8u6EoCZBruimWhqlTA.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-fZ_fO4ej9uBmnBRbUGX2gA.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-H7ZkAuPJkXaAMU25wCFOuQ.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-WjXJ4JoVDAMjGDVi3Cjxww.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-4bS6jMhsgVepujEQ_hnwBA.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-6f467dlU-hB37SBavhJf4w.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-DORHYI-9D95yjcISX-9qwQ.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-rQo0Gu5fstcFllnE8OVDXw.jar
Apr 01, 2020 1:49:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-EXJeECQHq8aKn6cEqI5CVA.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-xbarSnpvKgp2Gs97T4aUJQ.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-c4K23cRcPbsHEtEAjuAiiw.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Z21AViAZBWeBkdv5PYZRCw.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-4A9BMTyevNfyGYNB8GUA8A.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-QSRaEdBLyMVYr6n0SLT0bA.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-rUesJrJANatPHpoRY0lrJw.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-0bpfeyGquTEjwknKLQY8bQ.jar
Apr 01, 2020 1:49:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-AiLmM3saVVj4quDkz1erww.jar
Apr 01, 2020 1:49:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Apr 01, 2020 1:49:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Apr 01, 2020 1:49:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash RrILil7W9H2EwlNOO_WZMw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RrILil7W9H2EwlNOO_WZMw.pb
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Apr 01, 2020 1:49:18 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Apr 01, 2020 1:49:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-31_18_49_19-12381208428607351589?project=apache-beam-testing
Apr 01, 2020 1:49:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-31_18_49_19-12381208428607351589
Apr 01, 2020 1:49:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-31_18_49_19-12381208428607351589
Apr 01, 2020 1:49:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T01:49:19.084Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Apr 01, 2020 1:49:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:25.032Z: Checking permissions granted to controller Service Account.
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:29.906Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:30.697Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:30.750Z: Expanding GroupByKey operations into optimizable parts.
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:30.795Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:30.936Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:30.975Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:31.010Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:31.043Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:31.544Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 1:49:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:31.632Z: Starting 5 workers in us-central1-f...
Apr 01, 2020 1:49:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-04-01T01:49:45.857Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 01, 2020 1:49:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:57.179Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 1:49:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:49:57.215Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Apr 01, 2020 1:50:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:02.717Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Apr 01, 2020 1:50:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:17.286Z: Workers have started successfully.
Apr 01, 2020 1:50:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:17.319Z: Workers have started successfully.
Apr 01, 2020 1:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:53.809Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Apr 01, 2020 1:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:53.992Z: Cleaning up.
Apr 01, 2020 1:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:50:54.076Z: Stopping worker pool...
Apr 01, 2020 1:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:52:45.120Z: Autoscaling: Resized worker pool from 5 to 0.
Apr 01, 2020 1:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-04-01T01:52:45.165Z: Worker pool stopped.
Apr 01, 2020 1:52:54 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-31_18_49_19-12381208428607351589 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 35ba3c24-3779-44f3-beed-80c62c3d7bda and timestamp: 2020-04-01T01:52:54.657000000Z:
Metric: Value:
read_time 19.112
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 49.578 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 27s
101 actionable tasks: 78 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/5fr63w4z5d6om
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #329
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/329/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9638] Strengthen worker region & zone options tests.
[boyuanz] [BEAM-9454] Add Deduplication PTransform
------------------------------------------
[...truncated 280.13 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 31, 2020 6:52:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 6:52:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 6:52:56 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 6:52:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 6:52:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 31, 2020 6:52:57 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-uE_SxJxXfWLz5DCA0zUT5Q.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-gSjNOc0DgLr38rd3n9N1AA.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-cUmX-crekGJd2W67WcClxg.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-5FjEmSFrPJmkMPSrDhN0zA.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-qWnOJemX-4iGUAdvm344Rg.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-BPxO_ygdop0ChGXYD95cfw.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-7FKBgpBLTPKElkhpgiTLmw.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-A1e6IgvrB06j1Isc2bK9oA.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-4pPwWZT4GHM-MP_nRK0_FQ.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-mS7-rAz4o4MFXyViAbNbBQ.jar
Mar 31, 2020 6:52:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-TXV7Lw5UI4CQMCwdtoYVRw.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-Pq0atcjP-mcUhbc6x8NRpg.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-oVkXM258gEmUY_QtpnSYGw.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-0kCgNnGzko97CLHqZhx-Kw.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-q0ZPjL3FdMXl1xKp1BGMmg.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-p8KrOWKQj1Vf8qkzVZirow.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-ScGb78ZD-ZHHt1mi0v4JoA.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-2AQf8INuX9804YUhCCLklA.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-yGhwzE5PlbMnQLxxbCmZBQ.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-LGz3Ry_0G9xJ3Kq6VLM6xg.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-h8cQAjq_6rRhSGaBfRLvzQ.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-7-TQuc-jhLs_2ST71_iFDw.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-41E-R6P0kIFVuid2-jxYGg.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-nAMslJkYlHk8r7TLzJArBw.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-JMuVLlznSaQnPr-nEuOW8A.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-s6O_eS5eBzn-NYxGhC-sIQ.jar
Mar 31, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-cP_1HIgW6gCbKIj-m8CR5A.jar
Mar 31, 2020 6:53:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 31, 2020 6:53:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 31, 2020 6:53:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 31, 2020 6:53:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 31, 2020 6:53:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 31, 2020 6:53:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 31, 2020 6:53:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 6jG5gYBlOWSjPESgFRXW9w> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6jG5gYBlOWSjPESgFRXW9w.pb
Mar 31, 2020 6:53:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 31, 2020 6:53:04 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 31, 2020 6:53:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-31_11_53_04-2135181064097878218?project=apache-beam-testing
Mar 31, 2020 6:53:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-31_11_53_04-2135181064097878218
Mar 31, 2020 6:53:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-31_11_53_04-2135181064097878218
Mar 31, 2020 6:53:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T18:53:04.142Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 31, 2020 6:53:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:08.044Z: Checking permissions granted to controller Service Account.
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:12.725Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.486Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.525Z: Expanding GroupByKey operations into optimizable parts.
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.565Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.807Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.841Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.874Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:13.908Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:14.310Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 6:53:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:14.397Z: Starting 5 workers in us-central1-a...
Mar 31, 2020 6:53:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T18:53:37.992Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 31, 2020 6:53:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:53:41.697Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 6:54:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:54:00.941Z: Workers have started successfully.
Mar 31, 2020 6:54:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:54:00.966Z: Workers have started successfully.
Mar 31, 2020 6:54:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:54:42.187Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 6:54:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:54:42.385Z: Cleaning up.
Mar 31, 2020 6:54:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:54:42.476Z: Stopping worker pool...
Mar 31, 2020 6:56:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:56:15.334Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 31, 2020 6:56:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T18:56:15.427Z: Worker pool stopped.
Mar 31, 2020 6:56:21 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-31_11_53_04-2135181064097878218 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): abbb7273-37e6-49a2-be76-48b6ab1d429a and timestamp: 2020-03-31T18:56:21.966000000Z:
Metric: Value:
read_time 22.114
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 34.583 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/reagixxe5m5ii
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #328
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/328/display/redirect>
Changes:
------------------------------------------
[...truncated 279.93 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 12:48:45 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 12:48:46 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 31, 2020 12:48:46 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 31, 2020 12:48:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 31, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 31, 2020 12:48:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-84EAfo94L-R1UNcUZffqMg.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-xh_qFfBG2r2hbxSIGl1--A.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-8JUtR9Ff7fN2d5GUOwAnsQ.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests--a8tqnr_u5lRM0rd0-8EPQ.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-yiL_vciJUev-vbZBPm4N9w.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-jk1Q388q43Otc5KvtJ5xoA.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-q1VPjsvDEoejLjVgrJUSyg.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-kTosalnjv4ql_eYcPNGnUA.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-m9SQ-xbwQMGoECkLdzyY1g.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-ahIoM8sFCuvMZPiloYnpXg.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-l1CLcX75SzJupDRqx5FvvA.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-f8TXcM4-RYqlwFtUKOtsmg.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-4XPzaN6-sxtDtD2u06Kmgw.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-Xzv06kM5eMu4bheRQKerJA.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-5W7Br3aMdlFMtOq4vvJsPQ.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-lLguo2s8U1yzKDoaHkhROw.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-O0VtSTGlmytIIj_c1IuAjA.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-6EdirEKAP2gWCaJZoQpVgw.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-3zO5ngnw0haA6TOroyTIjw.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-QRDZZmmQT6Og97L1bL3eDQ.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-Qc2Ed8bYGZuparf8H_621Q.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-X66M7c8_HhDHiC94fm7j7w.jar
Mar 31, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-foo3xxDslPLOACABNjNeiA.jar
Mar 31, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-7eAnhneQnfLXyh6-z7ZvVA.jar
Mar 31, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-r4Yp775GySV-_-F-RDDnpg.jar
Mar 31, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-241s1RZEJWvBHweW57Sr3w.jar
Mar 31, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-cx2S3495hW_F6J6kILnvCw.jar
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 31, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12082 bytes, hash fkotH8YwL-g8Md7NEcllIA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fkotH8YwL-g8Md7NEcllIA.pb
Mar 31, 2020 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 31, 2020 12:48:50 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 31, 2020 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-31_05_48_50-7171540065309907277?project=apache-beam-testing
Mar 31, 2020 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-31_05_48_50-7171540065309907277
Mar 31, 2020 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-31_05_48_50-7171540065309907277
Mar 31, 2020 12:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T12:48:50.711Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 31, 2020 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:53.756Z: Checking permissions granted to controller Service Account.
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:57.690Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.361Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.398Z: Expanding GroupByKey operations into optimizable parts.
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.425Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.497Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 31, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.522Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 31, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.555Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 31, 2020 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.590Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 31, 2020 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:58.976Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:48:59.052Z: Starting 5 workers in us-central1-f...
Mar 31, 2020 12:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T12:49:09.278Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 31, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:49:25.624Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:49:25.660Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Mar 31, 2020 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:49:31.097Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 12:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:49:52.276Z: Workers have started successfully.
Mar 31, 2020 12:49:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:49:52.304Z: Workers have started successfully.
Mar 31, 2020 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:50:32.329Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:50:32.547Z: Cleaning up.
Mar 31, 2020 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:50:32.641Z: Stopping worker pool...
Mar 31, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:52:30.996Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 31, 2020 12:52:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T12:52:31.070Z: Worker pool stopped.
Mar 31, 2020 12:52:37 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-31_05_48_50-7171540065309907277 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d22ed021-1c4e-4377-ab80-a77bcc064fa0 and timestamp: 2020-03-31T12:52:37.645000000Z:
Metric: Value:
fields_read 4375276.0
read_time 21.737
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 0.739 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 11s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/2hjqis2bm2tko
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #327
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/327/display/redirect>
Changes:
------------------------------------------
[...truncated 280.68 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 31, 2020 7:02:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 7:02:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 7:02:32 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 7:02:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 7:02:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 31, 2020 7:02:33 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-eu1CHXPpFT71Ja2PAWVghg.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-Y-uZAOp3d7uHNNsthOpDOw.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-4A747JUMF9TmJyriiB2Gyw.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-4V9Z-5qWoNx5gLpwUOcTIw.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-vGGNunukJ0m4aczMcjz31w.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT--f7utK21ZQDyEcGlRGTdUQ.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-2PvzMfRhG0zWNJykXfKA3w.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-kZeNJtP-uI3GNT96FpwBCA.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-j8-WgEt8Len5UkC47Sm0Cg.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-VqJ1CqXTm9IOVfmtu09xjg.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-Gy5946w36KubgecHlBZt4Q.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-UVvh9s6gj7aZ-YTCJ9L2_A.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-TxFCLnkPBj6lVVMH4N_TTA.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-P9vF7o1ptwZxQ7uusa1Uiw.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-4KdWnSogrgJ8A-zaFkLLHg.jar
Mar 31, 2020 7:02:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-axdrySjt-PMnn0qQVdvzNg.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-ogxAvxilqqfwkLPGWtg4qQ.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-5uV33c4CsMdjC7dZXUfGfA.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-t967vX3P5XbzuZ9y6sy7Ng.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-roNBwsgat6sztrnkfknKbA.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-cIqXMVfnUUO35wZ4sMQvig.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-rtM-lVQ7B069T_xDtMFeVA.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Nk6QuU6gcB2t7cfEbngG0A.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-6Zx2NNmmKfTAcxf0ehLWeQ.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-PSsrCJrRuFgTQD4MuPErCA.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-0rlH-ug-5t0MLFkfUfRNRQ.jar
Mar 31, 2020 7:02:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-YSQQaflvlkprw9-q7E0lCQ.jar
Mar 31, 2020 7:02:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 31, 2020 7:02:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash JLL38JxJRdOGGi_w4KpUww> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-JLL38JxJRdOGGi_w4KpUww.pb
Mar 31, 2020 7:02:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 31, 2020 7:02:38 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 31, 2020 7:02:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-31_00_02_38-10377449722296112624?project=apache-beam-testing
Mar 31, 2020 7:02:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-31_00_02_38-10377449722296112624
Mar 31, 2020 7:02:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-31_00_02_38-10377449722296112624
Mar 31, 2020 7:02:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T07:02:38.626Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 31, 2020 7:02:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:42.148Z: Checking permissions granted to controller Service Account.
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:48.229Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:48.955Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:48.996Z: Expanding GroupByKey operations into optimizable parts.
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.051Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.133Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.160Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.180Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 31, 2020 7:02:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.219Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 31, 2020 7:02:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.639Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 7:02:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:02:49.713Z: Starting 5 workers in us-central1-f...
Mar 31, 2020 7:03:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T07:03:03.721Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 31, 2020 7:03:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:03:19.768Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 7:03:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:03:19.804Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Mar 31, 2020 7:03:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:03:25.254Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 7:03:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:03:38.483Z: Workers have started successfully.
Mar 31, 2020 7:03:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:03:38.510Z: Workers have started successfully.
Mar 31, 2020 7:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:04:22.405Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 7:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:04:22.596Z: Cleaning up.
Mar 31, 2020 7:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:04:22.675Z: Stopping worker pool...
Mar 31, 2020 7:06:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:06:15.746Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 31, 2020 7:06:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T07:06:15.794Z: Worker pool stopped.
Mar 31, 2020 7:06:21 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-31_00_02_38-10377449722296112624 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): d33678aa-4294-4189-8f64-687dc92ae673 and timestamp: 2020-03-31T07:06:22.031000000Z:
Metric: Value:
fields_read 4375276.0
read_time 24.309
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 58.044 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/zarni24jmmybc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #326
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/326/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9446] Retain unknown arguments when using uber jar job server.
[kcweaver] Enable '--option=value' and '-option' syntax.
[rohde.samuel] Address leftover styling comments from PR/10892
[rohde.samuel] address comments
[lostluck] accept generated metrics
[github] Use split instead of rsplit
[lostluck] Remove mType and move type urns to urnToType
[lostluck] add missing pcollection payload
[kawaigin] [BEAM-7923] Fix datatable on notebook reloading
[robinyqiu] Clean up code in ZetaSQLDialectSpecTest
------------------------------------------
[...truncated 280.10 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 31, 2020 12:58:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 31, 2020 12:58:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 31, 2020 12:58:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 31, 2020 12:58:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 31, 2020 12:58:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 31, 2020 12:58:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-j57fUmkAPuGPesT0R6JD9Q.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-dqd-ZvwHFl4iusttIqfOyQ.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests--sugeKVUPBkAcZwSdGpV5A.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-y_oZ4Zc8sWjL0ctfuOxk0Q.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-GEeDM_fOJTQnISSFaW254g.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-80Deefv3qWMz2oBQGJHXYw.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-rEhm9eLcs3oZHkewGuuMKQ.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-9VBSyIEKtBixbeIgBCNaJg.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-BEpGo48fpYUutqFTHxGLAQ.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-O5a0XBJafk_5QlcmIyYsgg.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-I96YmoJbVxds1ySFa402Qg.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-x_p2PvOE1r5HiKgjmSiKbA.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-8V5LDqi3tdcJUb5ZT94m7w.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-2Z65bpWcqIls30710J-oHA.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-wPxGL7uZlSH1129xbPYFww.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-yNotnIxvmE875Yawi9-o1Q.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-t0v0t8YnxGQXHv_Muclolw.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-xPo4d3tN8YxZQMBByYttVg.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-2RRihhnA_rKEh9hF81bipA.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-zZwNyDMuALpI50f4-8O11g.jar
Mar 31, 2020 12:58:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-q-H9eTF-c0E65nFzIMrbGg.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-Iy5NnQhyARr8TdW1GGEohg.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-qUXMDDethc9j7_ua2C8ukg.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-7CpiXwY_8LXAi-3IeNL7Lw.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-fn9nEt5yyXHos6CG6g0n_g.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-h2AVTNidRPudeU1etG3WVw.jar
Mar 31, 2020 12:58:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-Q7wl2CB0_I1WS-LvCBe83g.jar
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 31, 2020 12:58:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 31, 2020 12:58:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 9UBdRVYNp1yKL1RkM48uUQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-9UBdRVYNp1yKL1RkM48uUQ.pb
Mar 31, 2020 12:58:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 31, 2020 12:58:25 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 31, 2020 12:58:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-30_17_58_25-1121238222124050549?project=apache-beam-testing
Mar 31, 2020 12:58:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-30_17_58_25-1121238222124050549
Mar 31, 2020 12:58:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-30_17_58_25-1121238222124050549
Mar 31, 2020 12:58:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T00:58:25.912Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 31, 2020 12:58:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:29.975Z: Checking permissions granted to controller Service Account.
Mar 31, 2020 12:58:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:35.348Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.007Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.051Z: Expanding GroupByKey operations into optimizable parts.
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.093Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.175Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.215Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.251Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.283Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.699Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 12:58:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:58:36.789Z: Starting 5 workers in us-central1-f...
Mar 31, 2020 12:59:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-31T00:59:02.390Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 31, 2020 12:59:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:59:06.579Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 31, 2020 12:59:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:59:22.479Z: Workers have started successfully.
Mar 31, 2020 12:59:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T00:59:22.604Z: Workers have started successfully.
Mar 31, 2020 1:00:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T01:00:02.839Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 31, 2020 1:00:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T01:00:03.024Z: Cleaning up.
Mar 31, 2020 1:00:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T01:00:03.092Z: Stopping worker pool...
Mar 31, 2020 1:01:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T01:01:42.588Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 31, 2020 1:01:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-31T01:01:42.742Z: Worker pool stopped.
Mar 31, 2020 1:01:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-30_17_58_25-1121238222124050549 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0100474c-f75d-461b-9a7b-e13421a926c3 and timestamp: 2020-03-31T01:01:49.922000000Z:
Metric: Value:
read_time 22.384
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 37.715 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 45s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/fngqfu6qvc2jg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #325
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/325/display/redirect?page=changes>
Changes:
[robertwb] Side input signals for legacy worker.
[github] Update documentation
[apilloud] [BEAM-9512] Map anonymous structs to schema
------------------------------------------
[...truncated 278.85 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 30, 2020 7:04:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 30, 2020 7:04:02 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 30, 2020 7:04:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 30, 2020 7:04:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 30, 2020 7:04:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-B1jWW_qhh2Tq-Plmc6t_ig.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-dcGP96yHVPNIJYN9I2SS-Q.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-e-AJii_8b2JOug2tnRMBrg.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-AmZhTqpKUOL3a17Y_L85Xw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-UTf4rYTYlWiC4y4wxI7I9Q.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-up_IYO29O877F4qSU8N95w.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-jTwcgaAP4C3DOGOaxNZhyQ.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-PRwO-JgHWX0HW4PxUgAr_g.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-OpFmHgoF2zvIsglbBxyIrA.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-uGPeIyPjALhor1roF4-AZw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-G1pW8tqqtDneX2PEeZXwMg.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-2AE2_cyhGPFXc9U2CBKgBQ.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-OXK-QnS7XBoe-SlkLobuFw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-10_Sl4HLCUSJGX_9GjXCpw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-fHQV4nNyhuX30iMB8wukgQ.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-FUqCgMft5KKPl-l0nJBZpQ.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT--GW0rafzck6gXKb8DZCxow.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-aSpGkSs4GL16JbUDYvPajw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-uSraHHq3aAPU48sjEJOt6w.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-9FzxYYM3ieCCPE6PO2tqKQ.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-SH7KhF5Ec7-f2bm_RE5G9w.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-xhJkn6LZi4DW_doXPu0j0g.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-g5xOrx6LANIrhBEspT7DdA.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-Kzw7Hm29j_1tLAXxqXvesw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-l-HwC-snAjIi1lGHn0TLAw.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-uB9Hmzss0qRbyv1bt-Auyg.jar
Mar 30, 2020 7:04:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-LYQWQ29HBzuz_wBXzNjYIg.jar
Mar 30, 2020 7:04:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 30, 2020 7:04:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash rql64d2TR1tX1f3dwI_4Eg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-rql64d2TR1tX1f3dwI_4Eg.pb
Mar 30, 2020 7:04:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 30, 2020 7:04:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 30, 2020 7:04:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-30_12_04_07-1128434183849612156?project=apache-beam-testing
Mar 30, 2020 7:04:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-30_12_04_07-1128434183849612156
Mar 30, 2020 7:04:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-30_12_04_07-1128434183849612156
Mar 30, 2020 7:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T19:04:07.584Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 30, 2020 7:04:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:11.376Z: Checking permissions granted to controller Service Account.
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:16.926Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.550Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.590Z: Expanding GroupByKey operations into optimizable parts.
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.616Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.699Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.742Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.779Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:17.817Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:18.233Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 7:04:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:18.317Z: Starting 5 workers in us-central1-a...
Mar 30, 2020 7:04:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T19:04:26.655Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 30, 2020 7:04:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:44.354Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 7:04:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:44.389Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Mar 30, 2020 7:04:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:04:49.922Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 7:05:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:05:03.477Z: Workers have started successfully.
Mar 30, 2020 7:05:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:05:03.515Z: Workers have started successfully.
Mar 30, 2020 7:05:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:05:47.634Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 7:05:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:05:47.833Z: Cleaning up.
Mar 30, 2020 7:05:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:05:47.918Z: Stopping worker pool...
Mar 30, 2020 7:07:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:07:33.894Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 30, 2020 7:07:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T19:07:33.948Z: Worker pool stopped.
Mar 30, 2020 7:07:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-30_12_04_07-1128434183849612156 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ad6ff947-780a-44ed-8697-b60842f62e8a and timestamp: 2020-03-30T19:07:41.531000000Z:
Metric: Value:
fields_read 4375276.0
read_time 28.042
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 3 mins 47.649 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 35s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/gzkzja6hr24xo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #324
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/324/display/redirect>
Changes:
------------------------------------------
[...truncated 281.96 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 30, 2020 12:53:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 12:53:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 12:53:18 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 12:53:18 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 30, 2020 12:53:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 30, 2020 12:53:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 30, 2020 12:53:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 30, 2020 12:53:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 30, 2020 12:53:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-FJaQFT7ylj0ogk_9dfMHdA.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-c3k_siPIIKE1MxBfTIOZqA.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-RQAwTVHZjPbpiM3iFpfI3Q.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-E8pCp8OaJFtENIc1PR6W8w.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-B5IQWn901QuRxZVAoFevCA.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-od0okditbDGjelJs2Rq46Q.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-b-VrvZxkbK9rBABx_FpYHg.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-OfzK9caMnGITOCvnbpeWKg.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-qYpkAr2RYn1-8zFlusUnFQ.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-BuOCdDZDnYcl-TZTSuCgyQ.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-P3FXXcEGUn2zHc7RKJuTJA.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-bZi2nGcFy99Zo1vRFdfL3w.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-HJklbqeKedkWDMXdugGt8A.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-dtLjDF1Ps4wpGcK72vLECQ.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-CdxA6YC_s0_Bv1vodXBknQ.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-0kkbO7eRhTuyMbqtV01zIA.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-_pnGfpS28DgGvzW4p-2k5w.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-PzTEYAkTaSXIMivPx6qKIg.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-8vrn3UaW7dBni8hsr_ru-g.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-kqZd50pfokUb0aosqsLg2w.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-s4Wa_F1W9SPAm8QWF_F9Wg.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-J3c1m1Mmhf8t8IZQMPZuuw.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-KFlzsIyNE5My1Vv4yHsu2Q.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-1TbOJOcAyVOzBff1_nc0Nw.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-ibsoHw_qDlCTtKJ_hlTdSQ.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-t5AQJe92hkSahke1eBHTBw.jar
Mar 30, 2020 12:53:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-S7Z7Vjo0q9iadjqhcReGKg.jar
Mar 30, 2020 12:53:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 30, 2020 12:53:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash K8mUwNDq5Ui6Qd1O6sizpg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-K8mUwNDq5Ui6Qd1O6sizpg.pb
Mar 30, 2020 12:53:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 30, 2020 12:53:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 30, 2020 12:53:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-30_05_53_24-13043536768682658136?project=apache-beam-testing
Mar 30, 2020 12:53:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-30_05_53_24-13043536768682658136
Mar 30, 2020 12:53:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-30_05_53_24-13043536768682658136
Mar 30, 2020 12:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T12:53:24.516Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 30, 2020 12:53:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:28.015Z: Checking permissions granted to controller Service Account.
Mar 30, 2020 12:53:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:32.488Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.226Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.269Z: Expanding GroupByKey operations into optimizable parts.
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.312Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.396Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.429Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.459Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.502Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:33.974Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 12:53:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:34.068Z: Starting 5 workers in us-central1-a...
Mar 30, 2020 12:53:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T12:53:52.622Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 30, 2020 12:53:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:58.836Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 12:53:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:53:58.873Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Mar 30, 2020 12:54:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:54:04.258Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 12:54:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:54:16.044Z: Workers have started successfully.
Mar 30, 2020 12:54:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:54:16.083Z: Workers have started successfully.
Mar 30, 2020 12:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:55:00.404Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 12:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:55:00.618Z: Cleaning up.
Mar 30, 2020 12:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:55:00.703Z: Stopping worker pool...
Mar 30, 2020 12:56:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:56:39.213Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 30, 2020 12:56:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T12:56:39.255Z: Worker pool stopped.
Mar 30, 2020 12:56:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-30_05_53_24-13043536768682658136 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a0ea422f-8732-430e-a181-b4d9dc06691d and timestamp: 2020-03-30T12:56:45.888000000Z:
Metric: Value:
read_time 26.546
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.038 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 36.118 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 29s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/cynsloac53jci
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #323
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/323/display/redirect>
Changes:
------------------------------------------
[...truncated 280.52 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 30, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 6:45:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 30, 2020 6:45:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-2k9gYBnGt67MGG6PM7NtMQ.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-yJzFw1aGAeFmMejIlOQHEA.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-E2s-grOb1pvWUCKOMQYGiQ.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-dffS93tSr-jKI34t2yH_cg.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-5v8JGeGmluzLJX7E2stOSw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-OZ74uX2jchja7dBFdEBruw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-GTdYwuUPS8AHRuZUBwd7gw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-IXG6S7YTBRIHtWvcHkjLcw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-4cQs2uj9U8VRUlwsVaYtAA.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-m26erWAdL53Z6F3qgVu7eQ.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-IsiYBvzzvmvNC4RkLp0a7Q.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-8FUYZTa5d9u2kJxWw6_pNA.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-di0PfIHGOKpIUOZlD1eZEQ.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-8bnHU832Zh8Oa9t2EI_Zsw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-bhdHZq8etMb81mTdQCwG9g.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-vGmhSySPMjUIxYtRRd2Xnw.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-ZNkFE9BwKuVekKObFaP-1g.jar
Mar 30, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-SPGQmqOVdD_eVgteEijWXw.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-4MdyQoCkRNuhmZ6_PuOEGQ.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-a50L6pzA7OhuxCnUpT6cLw.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-hhzuEGZo429C-idHYOPAXA.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT--pZzu5EklNBCbhMolBrrGA.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-biYhAlVbtgxLA4_knxdJCA.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-BfzqmTJL8NmL_EmorgrNbg.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-iO4ZNf2Q-CqUeNWwIcxPuA.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-uz5FUzPfriOZ6NxAT0bvJQ.jar
Mar 30, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-LVU1tFArPRGcLSwPG8k0AA.jar
Mar 30, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 30, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12085 bytes, hash mom_xWt3tOx_R4KyNxe5GA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mom_xWt3tOx_R4KyNxe5GA.pb
Mar 30, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 30, 2020 6:45:36 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 30, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-29_23_45_36-3261379860683094874?project=apache-beam-testing
Mar 30, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-29_23_45_36-3261379860683094874
Mar 30, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-29_23_45_36-3261379860683094874
Mar 30, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T06:45:36.412Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 30, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:40.104Z: Checking permissions granted to controller Service Account.
Mar 30, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:44.672Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.310Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.368Z: Expanding GroupByKey operations into optimizable parts.
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.388Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.486Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.527Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.566Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:45.601Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:46.013Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:45:46.097Z: Starting 5 workers in us-central1-f...
Mar 30, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:46:09.522Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:46:09.554Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Mar 30, 2020 6:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T06:46:09.677Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 30, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:46:14.971Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 6:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:46:31.744Z: Workers have started successfully.
Mar 30, 2020 6:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:46:31.784Z: Workers have started successfully.
Mar 30, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:47:09.329Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:47:09.540Z: Cleaning up.
Mar 30, 2020 6:47:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:47:09.639Z: Stopping worker pool...
Mar 30, 2020 6:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:48:42.849Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 30, 2020 6:48:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T06:48:42.914Z: Worker pool stopped.
Mar 30, 2020 6:48:48 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-29_23_45_36-3261379860683094874 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 07b15c4e-fbd2-4338-a5aa-ef856ae04929 and timestamp: 2020-03-30T06:48:48.573000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.676
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 29.043 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 28s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/irtclr6oikz3m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #322
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/322/display/redirect>
Changes:
------------------------------------------
[...truncated 279.64 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 30, 2020 12:45:16 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 30, 2020 12:45:17 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 30, 2020 12:45:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 30, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 30, 2020 12:45:18 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-00WtqU__yfkuzijB663-CA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-ggPJUe5XFOpYubfZeQWBLw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-ZenHi3ayXSuIAJFF6YHiYw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-jQLfKKjnhG4IBZ3IyK0sgA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-ui16AQI2YLzvbwxjWhds6A.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-qPS7NDwACo73s37mVGRTHw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-gqgOFoYlBDYWB_n59icVSQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-ZaG1qyk3V6HmOomtOP8yuw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-Sh2LhJwMg0crrMAkfhpdmw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-bKl-AeXIbjY4b5u9ySPLRA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-MnQy75VGqpLj7tbyileliA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-vuV1ror_MxOu-xKltoWVPA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-qpY0LIDZbxRSAjfFN6H2yQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-q4l-Xt1cqh0tcfe3YFblNg.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-M7wQdBHDu0eIlotFiPRK_A.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-EiqB0iK5yghwoTp71AIHoQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-3JSyP6Z3e2szKfnUeKF2Uw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-ky7hqyQy41zU-G0vRxFhjA.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-AYVIQgWWKqqTeRkfG-0IDQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-yZbJDbgjbcNNaPkXCjC6gw.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Wg-iYakbic26p2WJr5N6QQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-x3B_7soOwZB4y1aN8HVXzg.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-MrXuKQ8HSrzwjy9V175L-A.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-eJSA4ZTKol04XgfQsJL78w.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-uIxLAWARfaLB9-7JRQ9c_A.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-p0roZNF9XiIcP6Di7m6MXQ.jar
Mar 30, 2020 12:45:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-vK11H-EcATq6XnaHBBCd2w.jar
Mar 30, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash IC_cmKpSqLNXCW1Gc5m1vw> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IC_cmKpSqLNXCW1Gc5m1vw.pb
Mar 30, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 30, 2020 12:45:22 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 30, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-29_17_45_22-6424068060087684832?project=apache-beam-testing
Mar 30, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-29_17_45_22-6424068060087684832
Mar 30, 2020 12:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-29_17_45_22-6424068060087684832
Mar 30, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T00:45:22.347Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 30, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:26.159Z: Checking permissions granted to controller Service Account.
Mar 30, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.202Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 30, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.776Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 30, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.810Z: Expanding GroupByKey operations into optimizable parts.
Mar 30, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.842Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.914Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.939Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.958Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:31.982Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:32.355Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:32.415Z: Starting 5 workers in us-central1-f...
Mar 30, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:45:59.147Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 30, 2020 12:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-30T00:46:02.148Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 30, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:46:22.049Z: Workers have started successfully.
Mar 30, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:46:22.070Z: Workers have started successfully.
Mar 30, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:47:08.120Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 30, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:47:08.278Z: Cleaning up.
Mar 30, 2020 12:47:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:47:08.347Z: Stopping worker pool...
Mar 30, 2020 12:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:48:39.573Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 30, 2020 12:48:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-30T00:48:39.615Z: Worker pool stopped.
Mar 30, 2020 12:48:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-29_17_45_22-6424068060087684832 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 763ed7e8-9fcc-4cfd-9c2a-3dbfe1a81999 and timestamp: 2020-03-30T00:48:45.326000000Z:
Metric: Value:
read_time 28.068
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 36.615 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 27s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/nlajtm2alaghq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #321
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/321/display/redirect>
Changes:
------------------------------------------
[...truncated 279.70 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 29, 2020 6:45:15 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 6:45:15 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 6:45:15 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 6:45:15 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 6:45:15 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 29, 2020 6:45:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-2wBBT-4cGIy6c6tBnBBD6g.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-Dm-g51lZghdRGRo7yZVSSw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-lEZEfk9OkjDqgD7FsNHPnw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-rC-nmgoDGblj5HRo9vHx2g.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-JI2bUTzijGdozcQQ011bCw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-uIYdKFcxaczIkYpUSS4nlg.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-6-h8SsR0qz0Usl_dav7ISg.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-ih8J_ke6hRNf4sZldj46Ew.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-BuWxwr_RbAR0AOmJEpa6dA.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-wmPASZEXxcsi8uwSUpciOw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-yPkKV1e9Q9EIXUdTJ4BAmA.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-QtHdy236z4anTaG-6uXz3g.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-dB5JUuCgPVtxZnKoLQmZQw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-CIVvj3a0UafyrfOfbwVJag.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-v6C57srkCRNidMjyG02Ccw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-JP72V7ifipu9dOCm0IBPYQ.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-NhOeP4QgCjTAc2BgOUZnEw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-XcWtcxIVbDRSoDELF5zmiw.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-hJtSy1CqUDGk3gwh-KdlYg.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-cmPySBXZtlwZB155OfeVJQ.jar
Mar 29, 2020 6:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-8bglqEZH2pdmGAXNJk_X4g.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-ZOLFNpKDzqhYBevV8LurYQ.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-Jg1JtBu2hYlFPTy3aOMDvg.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-XgcmyAld7Al5jpfI-rqbbQ.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-tLJQrGx-lhvulAS1Bv4yig.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-kZIjG6uNtt2LIgLGbnoq9Q.jar
Mar 29, 2020 6:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-IgxwKbueHg289mFdpb5aVQ.jar
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 29, 2020 6:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 29, 2020 6:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash pwr_gTpn7B1w_J7CgtAJ5Q> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pwr_gTpn7B1w_J7CgtAJ5Q.pb
Mar 29, 2020 6:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 29, 2020 6:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 29, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-29_11_45_20-9140531816662597517?project=apache-beam-testing
Mar 29, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-29_11_45_20-9140531816662597517
Mar 29, 2020 6:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-29_11_45_20-9140531816662597517
Mar 29, 2020 6:45:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T18:45:21.007Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 29, 2020 6:45:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:24.449Z: Checking permissions granted to controller Service Account.
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.125Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.756Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.798Z: Expanding GroupByKey operations into optimizable parts.
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.842Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.924Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.965Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:29.997Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 29, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:30.035Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 29, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:30.538Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 6:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:30.610Z: Starting 5 workers in us-central1-f...
Mar 29, 2020 6:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:57.445Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 6:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:45:57.480Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Mar 29, 2020 6:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T18:45:58.203Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 29, 2020 6:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:02.920Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 6:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:17.626Z: Workers have started successfully.
Mar 29, 2020 6:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:17.664Z: Workers have started successfully.
Mar 29, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:54.278Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:54.499Z: Cleaning up.
Mar 29, 2020 6:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:46:54.594Z: Stopping worker pool...
Mar 29, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:48:44.320Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 29, 2020 6:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T18:48:44.373Z: Worker pool stopped.
Mar 29, 2020 6:48:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-29_11_45_20-9140531816662597517 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 04508d04-8b9f-4d8b-9c81-8a008337d2e3 and timestamp: 2020-03-29T18:48:51.703000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.083
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 44.607 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 34s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/hl4jstgw6fpoi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #320
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/320/display/redirect>
Changes:
------------------------------------------
[...truncated 280.03 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 12:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 29, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-cRp3m_ldIl6I3o5VXmBI9w.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-lPhN2ZZg_09OFD4ECYG3ug.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-vL2R54WCZLuLqjA9LbyFWQ.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-rB1nPrx8szPeFxnGx20F8Q.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-x1OlPvfFeeHfX3ZPK3EJ2g.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-HVt98mBtUgbtVoTKoNCkew.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-q9tYVJQX0UQ9BRy-oE6cRg.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-oevegE1kPnevhlsdgjgGyw.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-nRrGVTS-73f4jhSSBsTE-g.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-AFOL8N00a6vtzTZS2LZETg.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-VPil5UOYTAVP-EMzELzrEQ.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-6jnJmtpfHDKmRM07gZVV6A.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-Rt4259MWhdH3Q0MenHbEOA.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-ffXgJI7xcLA5D2OH5mvdbA.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-tOEoD3CFRlfpVF9ebDON2Q.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-0UmAMiOOxRoCKbgFBy2eNA.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-lsIv5gNdQxD_mdYVTNYN7g.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-3zxp8yHSKaPz0ovq-m2zAg.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-pun5EhHygbbZnA4_ixnzag.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-GnKYRu2Vomazik8WI9Xong.jar
Mar 29, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-UdxzdIWLfaznhJn6sxyGIw.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-XclmVf08QBsjlxpWpVlgbA.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-tDbB25QoM_3dc1mPDjbiuw.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-GZqAkwArfqUvJSKq21EwrQ.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT--bTV5uWKKXdyGzOUZIggfg.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-zrVH2tR-xXvmzSRdM3qluQ.jar
Mar 29, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-jxOn7kiQZBCRkCrPTfsdjg.jar
Mar 29, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 29, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 3tqkvVwwmVetO1vyqXj8cA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3tqkvVwwmVetO1vyqXj8cA.pb
Mar 29, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 29, 2020 12:45:40 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 29, 2020 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-29_05_45_40-10011964927801896426?project=apache-beam-testing
Mar 29, 2020 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-29_05_45_40-10011964927801896426
Mar 29, 2020 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-29_05_45_40-10011964927801896426
Mar 29, 2020 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T12:45:40.172Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 29, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:52.092Z: Checking permissions granted to controller Service Account.
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:55.227Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:55.868Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:55.912Z: Expanding GroupByKey operations into optimizable parts.
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:55.952Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.028Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.060Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.099Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 29, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.131Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 29, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.602Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:45:56.692Z: Starting 5 workers in us-central1-f...
Mar 29, 2020 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T12:46:10.937Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 29, 2020 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:46:25.701Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:46:25.721Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Mar 29, 2020 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:46:31.148Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:46:47.728Z: Workers have started successfully.
Mar 29, 2020 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:46:47.770Z: Workers have started successfully.
Mar 29, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:47:26.546Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:47:26.723Z: Cleaning up.
Mar 29, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:47:26.803Z: Stopping worker pool...
Mar 29, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:48:48.016Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 29, 2020 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T12:48:48.065Z: Worker pool stopped.
Mar 29, 2020 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-29_05_45_40-10011964927801896426 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): af354b01-68a7-40c2-ab6d-f1ab39c63741 and timestamp: 2020-03-29T12:48:53.986000000Z:
Metric: Value:
fields_read 4375276.0
read_time 21.041
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 27.5 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 34s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/2f6cos7if2yg6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #319
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/319/display/redirect>
Changes:
------------------------------------------
[...truncated 281.09 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 29, 2020 6:47:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 6:47:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 6:47:28 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 6:47:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 6:47:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 29, 2020 6:47:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 29, 2020 6:47:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 29, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 29, 2020 6:47:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-bsNVieL8mVKNbwQbLNrslQ.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-MwWNYpGeoAQb8mEzve3GVA.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-AX17pGd-nkr2EmMDr93qFA.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-FpqZ5da-YaUb35hdaDQouQ.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-dpXuYXMKPsM7P1cEW2Sw5g.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-uIKMJp5igb_bkr8CnAApsw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-dRn4doiZIbl21i3KkzOIrw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-KiNANWgPer0Hgk3C2bvxKw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-efoKlQ7pSwUOb4XYAoTaXg.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-qFmmpO695uPY6ODKAc64pw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-Z2lLV9dmfnzJDWp9eHC4xw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-mR0g3DwkXr6Z4XQ83_UK0g.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-iQYJhLGm7rdMo9-4rPvedA.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-gTf4VbYFFtKx0D9d4QEGPw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-UL1rqZSApUsHN-mJ9w64Qg.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-vJ_j2RgL8WGLJ0sg6PHuQQ.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-BQnP_4RscFmsLyq3Dy0sDw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-6RojVeSJrilzyqMvlxqrtw.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-2uLeSGUyXcATVcrxWTUlcA.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-ya-bzaPCNnC5UbQ2MjZt6w.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-pUea_Hz_jIrqKCAHzntplQ.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-bd0ROpSkjYko-263mFMNjg.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-T92Z-Pj1UzQnyL0SRxVT5w.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-G0w5G2b4dBiR8joEBYgv9w.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-nlOyfDqnN5rfH2pRR93Rlg.jar
Mar 29, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-a1E74-zAc9lcAAkfpblz6A.jar
Mar 29, 2020 6:47:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-DhinjNWAv3KpabJTFGWfgA.jar
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 29, 2020 6:47:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 29, 2020 6:47:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash 0A2RpVR_PN3VBBOClGOgAg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0A2RpVR_PN3VBBOClGOgAg.pb
Mar 29, 2020 6:47:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 29, 2020 6:47:34 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 29, 2020 6:47:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-28_23_47_34-18289548195451753378?project=apache-beam-testing
Mar 29, 2020 6:47:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-28_23_47_34-18289548195451753378
Mar 29, 2020 6:47:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-28_23_47_34-18289548195451753378
Mar 29, 2020 6:47:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T06:47:34.981Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 29, 2020 6:47:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:38.632Z: Checking permissions granted to controller Service Account.
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:43.817Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.469Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.514Z: Expanding GroupByKey operations into optimizable parts.
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.547Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.622Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.652Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.676Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 29, 2020 6:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:44.698Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 29, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:45.041Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 6:47:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:47:45.108Z: Starting 5 workers in us-central1-f...
Mar 29, 2020 6:48:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T06:48:10.206Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 29, 2020 6:48:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:48:12.666Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 6:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:48:31.014Z: Workers have started successfully.
Mar 29, 2020 6:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:48:31.036Z: Workers have started successfully.
Mar 29, 2020 6:49:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:49:15.007Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 6:49:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:49:15.264Z: Cleaning up.
Mar 29, 2020 6:49:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:49:15.357Z: Stopping worker pool...
Mar 29, 2020 6:50:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:50:47.296Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 29, 2020 6:50:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T06:50:47.331Z: Worker pool stopped.
Mar 29, 2020 6:50:52 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-28_23_47_34-18289548195451753378 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 764475d6-6201-466a-a504-ea10cdb6be18 and timestamp: 2020-03-29T06:50:52.783000000Z:
Metric: Value:
read_time 25.926
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.042 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 35.183 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 3s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ydir65jwc3nuy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #318
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/318/display/redirect?page=changes>
Changes:
[github] [BEAM-9557] Fix timer window boundary checking (#11252)
------------------------------------------
[...truncated 279.87 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 29, 2020 1:09:14 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 29, 2020 1:09:15 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 29, 2020 1:09:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 29, 2020 1:09:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 29, 2020 1:09:16 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-k6lIrW9gJnGIJGfbUD2KPg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-U3yDgwl0lG9u2dPEaXjovA.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-Ipbs1MxoHg0NL2twRZ1OHg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-4j87iXXYNaQRepjkjoX1tA.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-g6pgxYTFQnFPgrp4YOfHPA.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-AEcdlzTEOHkg7UFMo8o6gQ.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-vuRsaNYvdpBAs7zvNVlOjw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-5LxBppSQtAj5EQDuKjAWuQ.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-gVO3QRquNcNttciI3ULWgw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-DT2pUI9wjqGZ2yJDO6Flmg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-TZobcSOl5Pg_sD2LJzNwuw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-8_v1dR3m7Krxp6VuxHWJPQ.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-PL-cq9H2ReEtzUH4RjhHTw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests--hiwm_q-uMxLkRVo_caZvw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-flZwKiaLys6QCG1tyYgM1Q.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-PQWpmSBzZZcDfDehOwcRtw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-4bbzY0x_-W6fTaaWpnlu3A.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-abj5J-B5WVmUvorDM7zJPQ.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-uIAlhn-8wAZs8tLNe6Y4fQ.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-kJgjqXTAJH406-Ey02UaGA.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-ZBw8YUQNRGmxM-5sM_ehPg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-6_BXUG690xrJKjr0ugKEKw.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-5xYam2gr48khVck3c_Uzzg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-gcNKv78CIK17F2ICZWp_bg.jar
Mar 29, 2020 1:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-Kem0uHTcQM1-TFAT_R-w-w.jar
Mar 29, 2020 1:09:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT--j6Lj1KEpc83K89g-1oz6w.jar
Mar 29, 2020 1:09:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-YCXB1RyarY7qh4aS1iQIfA.jar
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 29, 2020 1:09:19 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash TVb0N_X8Q_eV0Jv897avQA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-TVb0N_X8Q_eV0Jv897avQA.pb
Mar 29, 2020 1:09:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 29, 2020 1:09:20 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 29, 2020 1:09:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-28_18_09_20-10771002465901477682?project=apache-beam-testing
Mar 29, 2020 1:09:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-28_18_09_20-10771002465901477682
Mar 29, 2020 1:09:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-28_18_09_20-10771002465901477682
Mar 29, 2020 1:09:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T01:09:20.684Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 29, 2020 1:09:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:24.528Z: Checking permissions granted to controller Service Account.
Mar 29, 2020 1:09:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:29.042Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 29, 2020 1:09:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:29.854Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 29, 2020 1:09:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:29.901Z: Expanding GroupByKey operations into optimizable parts.
Mar 29, 2020 1:09:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:29.942Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.048Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.071Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.100Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.138Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.641Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 1:09:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:30.724Z: Starting 5 workers in us-central1-a...
Mar 29, 2020 1:09:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:54.606Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 1:09:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:09:54.637Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Mar 29, 2020 1:09:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-29T01:09:54.895Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 29, 2020 1:10:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:00.031Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 29, 2020 1:10:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:11.284Z: Workers have started successfully.
Mar 29, 2020 1:10:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:11.372Z: Workers have started successfully.
Mar 29, 2020 1:10:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:55.714Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 29, 2020 1:10:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:55.908Z: Cleaning up.
Mar 29, 2020 1:10:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:10:55.983Z: Stopping worker pool...
Mar 29, 2020 1:12:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:12:46.553Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 29, 2020 1:12:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-29T01:12:46.609Z: Worker pool stopped.
Mar 29, 2020 1:12:52 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-28_18_09_20-10771002465901477682 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 69780128-680a-4bca-a49d-aa08e35eece4 and timestamp: 2020-03-29T01:12:52.674000000Z:
Metric: Value:
fields_read 4375276.0
read_time 26.63
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 46.333 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 37s
101 actionable tasks: 66 executed, 35 from cache
Publishing build scan...
https://gradle.com/s/z4t6r435hzaoo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #317
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/317/display/redirect>
Changes:
------------------------------------------
[...truncated 279.95 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 28, 2020 6:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 28, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 28, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 28, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-CwvdDADT8wIl4cR3OVLuEg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-jOUljUp1qIuAgw4PhuvYZw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-ikW6IWfi8GPLIkRSW7y9Rg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-V8c0R1kIPOca3X4G91pV2w.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-zW-egOKrkRjbR7-Zxk9FMw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-qHap4rg79ZUReAJRg5WgHw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-n0YpLeByJ3MMAFXgOqCvMg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-6c8HSsR8FpOXZoBr4BR6IQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-S0KjvilsdSaLj8EpVWTMBQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-yhnj5BfqhsP5s3zRF_8z-w.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-6zfQA4wHB3RihXRiDHb5PQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-9rsb6JnuN47fH79U22jcWw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-HRjfTkT_EKYKNNY6I464Gw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-bb9Lcl4AsJjbM-p6LrwA_w.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-zDILpoztKaFnLM504e7eFA.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-b99K3ysQwY7-uPiN7o4Ckg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-dFOJpk8HNltXONzb6Dm2Rg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-Oyqu4DwpFvFuycQl5_ExZw.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-TtkiDLCxXR0zuj_uXzTGRQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-BEe8GODDP4rVhz_mtQ8eeg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-c1Of0SxqO0C1kHwx0fBaEA.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-y6zk45RrHZ0h_74xB53nxQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-i_Lv1UropdZJpaEjlEU3XQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-HJaq9dUCWcXGL3bsYJVrjg.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-9Bs2bhptJnY4lF_gGGs6dQ.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-DeKB-eNOfgUJMGajfhEe0g.jar
Mar 28, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-5YrKhRJ69uLICJF6bLF1Mw.jar
Mar 28, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash PhqZOjupVZioqsQA338aPA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PhqZOjupVZioqsQA338aPA.pb
Mar 28, 2020 6:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 28, 2020 6:45:41 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 28, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-28_11_45_41-16073578724029492083?project=apache-beam-testing
Mar 28, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-28_11_45_41-16073578724029492083
Mar 28, 2020 6:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-28_11_45_41-16073578724029492083
Mar 28, 2020 6:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T18:45:41.372Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 28, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:44.897Z: Checking permissions granted to controller Service Account.
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:49.686Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.448Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.496Z: Expanding GroupByKey operations into optimizable parts.
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.539Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.612Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.641Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.671Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 28, 2020 6:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:50.702Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 28, 2020 6:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:51.127Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 6:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:45:51.200Z: Starting 5 workers in us-central1-f...
Mar 28, 2020 6:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T18:45:55.850Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 28, 2020 6:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:46:17.021Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 28, 2020 6:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:46:33.890Z: Workers have started successfully.
Mar 28, 2020 6:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:46:33.923Z: Workers have started successfully.
Mar 28, 2020 6:47:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:47:09.637Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 6:47:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:47:09.827Z: Cleaning up.
Mar 28, 2020 6:47:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:47:09.905Z: Stopping worker pool...
Mar 28, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:48:33.406Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 28, 2020 6:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T18:48:33.447Z: Worker pool stopped.
Mar 28, 2020 6:48:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-28_11_45_41-16073578724029492083 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3006b2dc-0c2b-4fb7-bb3d-109f4eade303 and timestamp: 2020-03-28T18:48:40.075000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.342
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 12.166 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 23s
101 actionable tasks: 69 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/faiu6vlxsp25i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #316
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/316/display/redirect>
Changes:
------------------------------------------
[...truncated 279.34 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 28, 2020 12:48:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 28, 2020 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 28, 2020 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 28, 2020 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-Agxn2mX2mtz590HdgASzvg.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-A2PcMjwUXYuHstl2MkAHMg.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-eMUglW7f4aDSJh-J9smfHw.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-oP5WTsuc4ZrR00k6fbcZHQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-lIzLMnIOKMScrlZxLVOKNw.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-SN2d0k0zGCv1z33rBrBoSA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-x4trd-TiVbWMjtB78fiHiQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-8kGgpgTUjpJrmsc804k_jw.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-JMCXn6t7IRapWPZD3s14Hg.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-fF2AmTzZfzYaBQjuxUA-YA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-VetFQLphZcJ_D-MPrbfVIw.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-KacYG_SGJsWUdNp2QqIxlQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests--2KsOH6Nzgvmb5I2PvgXLQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-HEtv-XjvFaGC4DBom9mU3w.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-czBsM6P7IFblSAE6cO3uSQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-z4aT_L8kawVAfa5yXCwxzA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-gkS2BDD8Bz-YhFTn9dv8pQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-6c1lH3t5zdup1CprHTEZug.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-3iWSPnzmpKXrLnqaGsi4OA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-fsEhnLb6-ngW9u3sghK_xA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-XoNBH8jpgVKk5UpSLCzpCw.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-sJlpA_e0TNUuKujU5KAFWQ.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-yKZSsg8hp74gO_8rk2pMgA.jar
Mar 28, 2020 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-TsR0__IBs9Xuabe2xlbNLQ.jar
Mar 28, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-wOIni0-f_PFGqxlYXEgFXw.jar
Mar 28, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-WYmhuPT5tFPVLBrP4EAUNA.jar
Mar 28, 2020 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-B01vs_z6rAaLnoEgAkhKng.jar
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 28, 2020 12:48:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash fw5ht2PBfKdkpEX4KxCMCA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fw5ht2PBfKdkpEX4KxCMCA.pb
Mar 28, 2020 12:48:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 28, 2020 12:48:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 28, 2020 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-28_05_48_24-13296778796160396639?project=apache-beam-testing
Mar 28, 2020 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-28_05_48_24-13296778796160396639
Mar 28, 2020 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-28_05_48_24-13296778796160396639
Mar 28, 2020 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T12:48:24.817Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 28, 2020 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:30.837Z: Checking permissions granted to controller Service Account.
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.055Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.642Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.685Z: Expanding GroupByKey operations into optimizable parts.
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.712Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.778Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.801Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.825Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:38.849Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:39.151Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:48:39.211Z: Starting 5 workers in us-central1-f...
Mar 28, 2020 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T12:48:53.104Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 28, 2020 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:04.735Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 28, 2020 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:21.388Z: Workers have started successfully.
Mar 28, 2020 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:21.418Z: Workers have started successfully.
Mar 28, 2020 12:49:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:56.824Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 12:49:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:57.036Z: Cleaning up.
Mar 28, 2020 12:49:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:49:57.127Z: Stopping worker pool...
Mar 28, 2020 12:51:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:51:32.154Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 28, 2020 12:51:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T12:51:32.196Z: Worker pool stopped.
Mar 28, 2020 12:51:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-28_05_48_24-13296778796160396639 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 0cd04535-1fda-471d-a211-fefc48b847d9 and timestamp: 2020-03-28T12:51:38.423000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.032
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 27.662 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 34s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/opj2tzey2cy2s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #315
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/315/display/redirect>
Changes:
------------------------------------------
[...truncated 278.66 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 28, 2020 6:45:57 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 28, 2020 6:45:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 28, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 28, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-eL3sbJA51Qx3rUKUCpHlJQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-Q5XO92j4cnhkmFcrIa6jiQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-dX0HPeCnrKM33JwMYochzw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-fHJjCGYq5ZZO6_Bt7oGigg.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-WlQI1hswO57DTc3s-2d-zQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-6shjLTBaMk9GXVhGsNS2iw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-yeCQb4mGDfjqm6pATijVNw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-L_OCZ9MrFdIYpC9Tqfly1A.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-wWOLkyr0dflTaQ3_uDhw5g.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-ZNRidYwUihtxVfIRPmxZZA.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-GpJx0-nTkF5PE1wFBB9i4Q.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-0Fkma_7LOW65ruGF_eajaw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-We8SRFPG4Ax4Oh2oxayZpg.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-2BZq0X6I4lwA_Raf-FOnAg.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-E3brLeb_U32QrWnIhbC5-g.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-FYqGifz7lTrypjJHx7YOyA.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-f5JoYfOjfaG_OYjinUHOeA.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-weYCq1T1zkT2cbNCzvMgLw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-5M0OvF3PuqEx8sQh5jDuBw.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-IHp7oF-ZCWtzfn3vJVAG8g.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-seI3K9O55dl8vJsduZ-fYQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-07EKTzEWTcQlY4uywukWjA.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-c1tHaSIjKQnwoOWFqBEVQQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-jeYR8WZwDgjJnQ7Ag_SaaQ.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-Ss1pBMIIaHtX270uWDLEMg.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-KB_g1-bNXNfqZhDLk21w7g.jar
Mar 28, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-52l7-NELNbO91ckgVmbCEw.jar
Mar 28, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 28, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12083 bytes, hash jzsyJwq-NprWUY1UkQC2gA> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-jzsyJwq-NprWUY1UkQC2gA.pb
Mar 28, 2020 6:46:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 28, 2020 6:46:02 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 28, 2020 6:46:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-27_23_46_02-17791678083621520217?project=apache-beam-testing
Mar 28, 2020 6:46:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-27_23_46_02-17791678083621520217
Mar 28, 2020 6:46:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-27_23_46_02-17791678083621520217
Mar 28, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T06:46:02.514Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 28, 2020 6:46:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:06.169Z: Checking permissions granted to controller Service Account.
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:11.342Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.013Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.058Z: Expanding GroupByKey operations into optimizable parts.
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.084Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.159Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.195Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.227Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.251Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.644Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:12.756Z: Starting 5 workers in us-central1-a...
Mar 28, 2020 6:46:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:37.186Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 28, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T06:46:43.958Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 28, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:55.070Z: Workers have started successfully.
Mar 28, 2020 6:46:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:46:55.101Z: Workers have started successfully.
Mar 28, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:47:30.277Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:47:30.439Z: Cleaning up.
Mar 28, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:47:30.502Z: Stopping worker pool...
Mar 28, 2020 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:49:10.704Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 28, 2020 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T06:49:10.747Z: Worker pool stopped.
Mar 28, 2020 6:49:16 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-27_23_46_02-17791678083621520217 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8c0575b7-199a-45f4-ab5f-cc3e1ff35924 and timestamp: 2020-03-28T06:49:16.473000000Z:
Metric: Value:
fields_read 4375276.0
read_time 18.354
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 27.244 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 45s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/s6cwnjyqg7rhc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #314
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/314/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-4374] Update protos related to MonitoringInfo.
[github] Merge pull request #10883: [BEAM-9331] Add better Row builders
[github] [BEAM-8292] Portable Reshuffle for Go SDK (#11197)
------------------------------------------
[...truncated 279.50 KB...]
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 28, 2020 1:42:52 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 28, 2020 1:42:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 28, 2020 1:42:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 28, 2020 1:42:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-1wd_oGC9UBdvtHYzukaFow.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-HEE0JrOVoI5asem3evyEnQ.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-_HtX499-1f1IbDc9Rd2swQ.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-o9GemYyuGMbP9pPFdp7DHg.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-0Ld42advsxOJKk4e20Xplg.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-ovrzTYKr2g8XYc-u0SX-KQ.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-Sf_dE34yxtB_aMfH5D2orw.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-FAT80wb9UAJTjx1WkRIIbA.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-2yZeUMfc8XfpVj6R3rGPXA.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-rMaBz-9-td-blsSyAxAj_w.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-2nxXzKrFcPX7x6OTRMtWBA.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-nQXy9IG3PokGL7t5IuHY4w.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-KT1PboV-1MhtsXzxBEuU8g.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-92C2suOyKF1ZozugYFXRLw.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-TUy8jfDz0zCvGZXYHnwXrw.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-uMTHtveRvbkeyNIhkysKpg.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-1sn67T8YYZRdi_GnqnIWog.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-7LZlfSGTmcOOE9pf2e0P1A.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-E36SLzWO6uULLGrZkyUyVQ.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-QakXwZMNgjh_UnO2PiB21Q.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.8/37ca9a9aa2d4be2599e55506a6d3170dd7a3df4/commons-csv-1.8.jar to gs://temp-storage-for-perf-tests/loadtests/staging/commons-csv-1.8-kBnZnWBy9I_ZsfkJ8MRdJA.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-N4nn15pkroaUdXYwLtCSug.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-1FIWflZ6KcEr7no9B9mtfQ.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-9aF6a4oxJp_iYaoZ5uL95Q.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-6YSz6iF-bojTl2ZEwJKs2Q.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-va-5_bbiNt30PMZN2uufvA.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-qWP2uXDdJsko1jpEAfnTbg.jar
Mar 28, 2020 1:42:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-PoNTHRmQbrWsDgcOridQ9w.jar
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 177 files cached, 28 files newly uploaded in 2 seconds
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 28, 2020 1:42:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash r3XhAFLbYm8nznDa1A0fow> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-r3XhAFLbYm8nznDa1A0fow.pb
Mar 28, 2020 1:42:57 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 28, 2020 1:42:57 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 28, 2020 1:42:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-27_18_42_57-10872153470653661902?project=apache-beam-testing
Mar 28, 2020 1:42:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-27_18_42_57-10872153470653661902
Mar 28, 2020 1:42:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-27_18_42_57-10872153470653661902
Mar 28, 2020 1:42:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T01:42:57.812Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 28, 2020 1:43:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:01.898Z: Checking permissions granted to controller Service Account.
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.065Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.720Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.756Z: Expanding GroupByKey operations into optimizable parts.
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.790Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.876Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.899Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:07.999Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:08.029Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:08.503Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 1:43:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:08.575Z: Starting 5 workers in us-central1-a...
Mar 28, 2020 1:43:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-28T01:43:27.245Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 28, 2020 1:43:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:32.659Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Mar 28, 2020 1:43:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:50.284Z: Workers have started successfully.
Mar 28, 2020 1:43:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:43:50.311Z: Workers have started successfully.
Mar 28, 2020 1:44:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:44:31.014Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 28, 2020 1:44:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:44:31.442Z: Cleaning up.
Mar 28, 2020 1:44:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:44:31.506Z: Stopping worker pool...
Mar 28, 2020 1:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:46:06.496Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 28, 2020 1:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-28T01:46:06.540Z: Worker pool stopped.
Mar 28, 2020 1:46:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-27_18_42_57-10872153470653661902 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): de2931f2-9254-4d59-94b8-6856ccdcc043 and timestamp: 2020-03-28T01:46:12.948000000Z:
Metric: Value:
read_time 20.665
fields_read 4375276.0
Gradle Test Executor 683 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.016 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.015 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 3 mins 28.763 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/sh7vkvprjmkmy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #313
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/313/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-4150] Use explicit map for data channel coders.
[robertwb] [BEAM-4150] Don't window PCollection coders.
[github] Update the range for pyarrow
[github] Update the Go SDK roadmap for April 2020 (#11239)
------------------------------------------
[...truncated 279.08 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 27, 2020 7:16:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 7:16:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 7:16:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 7:16:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 27, 2020 7:16:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-pfuj0d8bvv9wwDmV6UVwhQ.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-aS0-yCptq9ckSCS9Lu4qcA.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-X4IXkF-iyYGDlxkkYm5Jww.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-exYTROagAzrAGn65C_RoTw.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-55abqD8AH6C_lh5QIaDOyw.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-O-UKbAT6U3-bZHH94gB6Kg.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-kOk0p_19LHIIrN7PfbpRZw.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-zo3-QoPXqEw8HUCAGZFE6Q.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-m4x01EKhypoOyFV_nszs8Q.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-qE2h3KR_-xelSBzZc88fMA.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-0B1yni4_XChjupn2gYqCzQ.jar
Mar 27, 2020 7:16:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-Tabt2OfWs7oD9wZdNbG1rw.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-iF9460VpjLk8es4a9bdePA.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-RL5v7SE86IFM1YOPuN2gxQ.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-lHoXC5oSLyPYJrYo-cRm0w.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-nu2dHzjofa7aGbZIFDlhKw.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-sOX0AkZVEznDFb0CfZ5eAA.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-APjqSMgJdB4d6CrMgDC_gA.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-pOdWTjvrS-P9SfD2ZIjtLA.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-YmRM_nsnWiqNS2oxcgINQQ.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-qbcm82kqomWZPoMhBOIopw.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-6vgg_jH819S_3ZSa7UgrZw.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-UnhGcOTQXnz_JnOr2dg1Eg.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-gjSwibNwMFPS77-TxdGiPg.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-rtEf6Ve1ksvmzFPe6e5SBg.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-wJm9s-Bz0KoOFLpeuHf7Wg.jar
Mar 27, 2020 7:16:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-iNkUBmYOFymSmYJCq9vG8w.jar
Mar 27, 2020 7:16:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 2 seconds
Mar 27, 2020 7:16:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash U1rcbl3ykfn2YPp0vxyoZg> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-U1rcbl3ykfn2YPp0vxyoZg.pb
Mar 27, 2020 7:16:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 27, 2020 7:16:56 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 27, 2020 7:16:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-27_12_16_56-11877592363028771214?project=apache-beam-testing
Mar 27, 2020 7:16:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-27_12_16_56-11877592363028771214
Mar 27, 2020 7:16:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-27_12_16_56-11877592363028771214
Mar 27, 2020 7:16:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T19:16:56.187Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 27, 2020 7:17:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:01.940Z: Checking permissions granted to controller Service Account.
Mar 27, 2020 7:17:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:08.423Z: Worker configuration: n1-standard-1 in us-central1-a.
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.059Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.101Z: Expanding GroupByKey operations into optimizable parts.
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.133Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.217Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.252Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.280Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.315Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.719Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 7:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:09.801Z: Starting 5 workers in us-central1-a...
Mar 27, 2020 7:17:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T19:17:18.366Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 27, 2020 7:17:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:17:39.306Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running step(s).
Mar 27, 2020 7:18:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:18:02.118Z: Workers have started successfully.
Mar 27, 2020 7:18:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:18:02.152Z: Workers have started successfully.
Mar 27, 2020 7:18:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:18:28.551Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 7:18:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:18:28.743Z: Cleaning up.
Mar 27, 2020 7:18:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:18:28.830Z: Stopping worker pool...
Mar 27, 2020 7:19:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:19:52.915Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 27, 2020 7:19:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T19:19:52.965Z: Worker pool stopped.
Mar 27, 2020 7:19:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-27_12_16_56-11877592363028771214 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 4ffc88f9-073d-4eb6-9621-37a4c52463fc and timestamp: 2020-03-27T19:19:59.257000000Z:
Metric: Value:
read_time 9.783
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 16.84 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 19s
101 actionable tasks: 68 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/khd7eey72ndls
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #312
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/312/display/redirect?page=changes>
Changes:
[alex] [BEAM-9605] BIP-1: Rename setRowOption to setOption on Option builder
[alex] [BEAM-9605] BIP-1: Rename setRowOption to setOption on Option builder
------------------------------------------
[...truncated 285.99 KB...]
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_3/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:142)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Mar 27, 2020 1:17:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 1:17:49 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 1:17:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Mar 27, 2020 1:17:49 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 1:17:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:156)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Mar 27, 2020 1:17:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 1:17:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Mar 27, 2020 1:17:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Mar 27, 2020 1:17:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Mar 27, 2020 1:17:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Mar 27, 2020 1:17:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Mar 27, 2020 1:17:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 27, 2020 1:17:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 27, 2020 1:17:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
WARNING: Skipping non-existent file to stage ${dataflowWorkerJar}.
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/classes/java/test> to gs://temp-storage-for-perf-tests/loadtests/staging/test-i9l_mYImCv2yRQsRNPPqAw.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-tests-MSAwyu54LJe9_vFtc0SNow.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-tests-cOB3mExpcW0wHXEY_Y89nw.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-u-HkFuERH-IUZL-LLIQwzQ.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-tests-5GFsuKwmD0DtTPYfLin4XQ.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.21.0-SNAPSHOT-fgzC1qlOvm4StJWtXn16SQ.jar
Mar 27, 2020 1:17:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT-tests-034zRcypHJOsz89oif10jA.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-VyI6nBGzQsT0YI6sYuEeew.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-tests-sgPSP3yYWiOoUJtlaDdI1Q.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.21.0-SNAPSHOT-50QI2MhNNeTpzoFJOY_i0w.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.21.0-SNAPSHOT-tQH2KLLsRjDPMVLZke4lLw.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.21.0-SNAPSHOT-1RPzleExxfG40Mk2N_5-7g.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.21.0-SNAPSHOT-6Yk_5RD3Lqga0HBnkPCICw.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-Yox4giJUs7-aIRafE9Y6bw.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.21.0-SNAPSHOT-tests-FcEkuaaQulysOYInkai9iw.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.21.0-SNAPSHOT-tests-9VxjQHrdFuMeZA-dr_OWeQ.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.21.0-SNAPSHOT-9ocPBeKQsAq6nY9p_nT1sg.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.21.0-SNAPSHOT-mbXgQkDJgqKpEoXRwZYygA.jar
Mar 27, 2020 1:17:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.21.0-SNAPSHOT-HjUlhmuxnfW00E1lkkBgsQ.jar
Mar 27, 2020 1:17:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.21.0-SNAPSHOT-hk5ko0ICObdhHy4KDJ72BQ.jar
Mar 27, 2020 1:17:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-unshaded-GF5zlHoBhBb7p5PQ_3Bkbg.jar
Mar 27, 2020 1:17:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT-tests-UUF9fHddfY2gmlBXr6w4og.jar
Mar 27, 2020 1:17:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.21.0-SNAPSHOT-DrlyLBSP70VFuonNQG7xxg.jar
Mar 27, 2020 1:17:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.21.0-SNAPSHOT--BGT72ZHesTMfBn9YH4W8w.jar
Mar 27, 2020 1:17:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.21.0-SNAPSHOT-SuvFiAWjQNDSAvmC4SuRLQ.jar
Mar 27, 2020 1:17:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.21.0-SNAPSHOT-rYnGEJd33y2S2KPW-5vmsw.jar
Mar 27, 2020 1:17:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.21.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.21.0-SNAPSHOT-PabrU-QG9KD9fpNm9Z4HkQ.jar
Mar 27, 2020 1:18:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 27 files newly uploaded in 7 seconds
Mar 27, 2020 1:18:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Mar 27, 2020 1:18:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Mar 27, 2020 1:18:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Mar 27, 2020 1:18:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Mar 27, 2020 1:18:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 27, 2020 1:18:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <12081 bytes, hash PujPuiiFD5NQElbgU7R1ZQ> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PujPuiiFD5NQElbgU7R1ZQ.pb
Mar 27, 2020 1:18:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.21.0-SNAPSHOT
Mar 27, 2020 1:18:05 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create
WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones
Mar 27, 2020 1:18:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-27_06_18_05-11859654544670276815?project=apache-beam-testing
Mar 27, 2020 1:18:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-03-27_06_18_05-11859654544670276815
Mar 27, 2020 1:18:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-03-27_06_18_05-11859654544670276815
Mar 27, 2020 1:18:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T13:18:05.336Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Mar 27, 2020 1:18:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:10.346Z: Checking permissions granted to controller Service Account.
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:16.094Z: Worker configuration: n1-standard-1 in us-central1-f.
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:16.751Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:16.915Z: Expanding GroupByKey operations into optimizable parts.
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:16.948Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.045Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.070Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.112Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Mar 27, 2020 1:18:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.146Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Mar 27, 2020 1:18:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.704Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 1:18:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:17.812Z: Starting 5 workers in us-central1-f...
Mar 27, 2020 1:18:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-27T13:18:23.786Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 27, 2020 1:18:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:18:48.017Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running step(s).
Mar 27, 2020 1:19:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:19:04.593Z: Workers have started successfully.
Mar 27, 2020 1:19:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:19:04.626Z: Workers have started successfully.
Mar 27, 2020 1:19:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:19:35.721Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Mar 27, 2020 1:19:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:19:36.144Z: Cleaning up.
Mar 27, 2020 1:19:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:19:36.225Z: Stopping worker pool...
Mar 27, 2020 1:21:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:21:10.698Z: Autoscaling: Resized worker pool from 5 to 0.
Mar 27, 2020 1:21:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-27T13:21:10.734Z: Worker pool stopped.
Mar 27, 2020 1:21:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-27_06_18_05-11859654544670276815 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 08ffd96b-1239-4856-82e3-f7c5b6cfe0c6 and timestamp: 2020-03-27T13:21:16.428000000Z:
Metric: Value:
read_time 13.5
fields_read 4375276.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.118 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.104 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 50.945 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 35s
101 actionable tasks: 80 executed, 21 from cache
Publishing build scan...
https://gradle.com/s/af3jqovxx33xo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org